2024-09-15T01:22:12.897 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T01:22:12.917 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T01:22:13.025 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045 branch: wip-bharath9-testing-2024-09-13-1444-quincy description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} email: skanta@redhat.com first_in_suite: false job_id: '7905045' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath9-testing-2024-09-13-1444-quincy ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd ms inject internal delays: 0.002 ms inject socket failures: 2500 osd_async_recovery_min_cost: 1 osd_object_clean_region_max_num_intervals: 1000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore zero block detection: true debug ms: 1 debug osd: 20 osd debug reject backfill probability: 0.1 osd debug verify cached snaps: true osd debug verify missing on start: true osd max backfills: 3 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: filestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd sloppy crc: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - slow request - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - mons down - mon down - out of quorum sha1: d75f23820806da622f561ee5ee7551015285993f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: enable_coprs: - ceph/el9 flavor: default sha1: d75f23820806da622f561ee5ee7551015285993f workunit: branch: wip-bharath9-testing-2024-09-13-1444-quincy sha1: d75f23820806da622f561ee5ee7551015285993f owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 2126 sha1: d75f23820806da622f561ee5ee7551015285993f sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath9-testing-2024-09-13-1444-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d75f23820806da622f561ee5ee7551015285993f targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOip3M6JR7Vt1szwnIKkcuL/R+M/J9h+XcrLEzyY876hi7Wm4U3zZJ3PQon/Nz2Ebd24yhRaubFM54sx1pmH90E= smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLg+RzEBGF5Qk4tevECApZMAdHoO9vuFaWkcyVtxY3AIZGMIqLtZFWVpyCt3ONGuLD/zo9ZGzf97c+iE9JsHgIE= smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIiChYAueR05V+t/G16CAcbzoHUNAoQSNL8H59l+nDyh7hko8Oss5r/8fq3E08hkxR7T3VLEHD/uwg3cHdPCYjo= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVudi/xyXepdtqOz3uqgucJ9F2fNOMFiGS369WpVBG82r/veIETy39qNvFZw6Ef3Do25lyGbgdpcVS/RaHLYiQ= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: bluestore_new_sharding: random chance_bluestore_reshard: 1 chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 8 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: c: 2 crush-failure-domain: osd k: 4 m: 3 name: shecprofile plugin: shec objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 400 write_append_excl: false teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_00:54:39 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T01:22:13.026 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa; will attempt to use it 2024-09-15T01:22:13.026 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa/tasks 2024-09-15T01:22:13.026 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T01:22:13.028 INFO:teuthology.task.internal:Checking packages... 2024-09-15T01:22:13.049 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd75f23820806da622f561ee5ee7551015285993f' 2024-09-15T01:22:13.049 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T01:22:13.049 INFO:teuthology.packaging:ref: None 2024-09-15T01:22:13.050 INFO:teuthology.packaging:tag: None 2024-09-15T01:22:13.050 INFO:teuthology.packaging:branch: wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-15T01:22:13.050 INFO:teuthology.packaging:sha1: d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:22:13.050 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-15T01:22:13.194 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1643.gd75f2382 2024-09-15T01:22:13.196 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T01:22:13.202 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T01:22:13.202 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T01:22:13.242 INFO:teuthology.task.internal:Saving configuration 2024-09-15T01:22:13.258 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T01:22:13.333 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T01:22:13.357 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 01:17:05.960059', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOip3M6JR7Vt1szwnIKkcuL/R+M/J9h+XcrLEzyY876hi7Wm4U3zZJ3PQon/Nz2Ebd24yhRaubFM54sx1pmH90E='} 2024-09-15T01:22:13.384 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 01:17:05.962287', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLg+RzEBGF5Qk4tevECApZMAdHoO9vuFaWkcyVtxY3AIZGMIqLtZFWVpyCt3ONGuLD/zo9ZGzf97c+iE9JsHgIE='} 2024-09-15T01:22:13.404 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 01:17:05.958865', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIiChYAueR05V+t/G16CAcbzoHUNAoQSNL8H59l+nDyh7hko8Oss5r/8fq3E08hkxR7T3VLEHD/uwg3cHdPCYjo='} 2024-09-15T01:22:13.421 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 01:17:05.961218', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVudi/xyXepdtqOz3uqgucJ9F2fNOMFiGS369WpVBG82r/veIETy39qNvFZw6Ef3Do25lyGbgdpcVS/RaHLYiQ='} 2024-09-15T01:22:13.422 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T01:22:13.435 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2024-09-15T01:22:13.435 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2024-09-15T01:22:13.435 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2024-09-15T01:22:13.436 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2024-09-15T01:22:13.436 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T01:22:13.602 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f03f4c1ee60>, signals=[15]) 2024-09-15T01:22:13.602 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T01:22:13.608 INFO:teuthology.task.internal:Opening connections... 2024-09-15T01:22:13.609 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T01:22:13.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:22:13.688 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-09-15T01:22:13.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:22:13.765 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2024-09-15T01:22:13.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:22:13.839 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-09-15T01:22:13.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:22:13.915 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T01:22:13.921 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-09-15T01:22:13.939 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-09-15T01:22:13.940 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-09-15T01:22:13.998 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-09-15T01:22:13.998 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-09-15T01:22:13.998 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-09-15T01:22:13.998 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-09-15T01:22:13.998 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-09-15T01:22:13.998 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-09-15T01:22:13.998 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T01:22:13.998 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-09-15T01:22:13.999 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-09-15T01:22:13.999 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T01:22:13.999 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-09-15T01:22:13.999 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T01:22:13.999 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T01:22:13.999 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T01:22:13.999 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-09-15T01:22:14.025 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-09-15T01:22:14.044 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-09-15T01:22:14.044 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-09-15T01:22:14.102 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-09-15T01:22:14.103 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-09-15T01:22:14.103 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-09-15T01:22:14.103 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-09-15T01:22:14.103 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-09-15T01:22:14.103 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-09-15T01:22:14.103 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T01:22:14.103 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-09-15T01:22:14.103 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-09-15T01:22:14.103 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T01:22:14.103 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-09-15T01:22:14.103 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T01:22:14.103 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T01:22:14.104 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T01:22:14.104 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-09-15T01:22:14.125 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2024-09-15T01:22:14.143 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2024-09-15T01:22:14.143 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2024-09-15T01:22:14.202 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2024-09-15T01:22:14.202 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="9" 2024-09-15T01:22:14.202 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2024-09-15T01:22:14.202 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2024-09-15T01:22:14.202 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="9" 2024-09-15T01:22:14.202 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el9" 2024-09-15T01:22:14.202 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T01:22:14.202 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2024-09-15T01:22:14.202 INFO:teuthology.orchestra.run.smithi152.stdout:LOGO="fedora-logo-icon" 2024-09-15T01:22:14.202 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T01:22:14.202 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2024-09-15T01:22:14.203 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T01:22:14.203 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T01:22:14.203 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T01:22:14.203 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2024-09-15T01:22:14.225 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-09-15T01:22:14.244 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-09-15T01:22:14.244 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-09-15T01:22:14.303 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-09-15T01:22:14.303 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-09-15T01:22:14.303 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-09-15T01:22:14.303 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-09-15T01:22:14.303 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-09-15T01:22:14.303 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-09-15T01:22:14.303 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T01:22:14.303 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-09-15T01:22:14.303 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-09-15T01:22:14.304 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T01:22:14.304 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-09-15T01:22:14.304 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T01:22:14.304 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T01:22:14.304 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T01:22:14.304 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-09-15T01:22:14.325 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T01:22:14.333 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T01:22:14.412 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T01:22:14.412 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-09-15T01:22:14.414 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-09-15T01:22:14.417 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2024-09-15T01:22:14.419 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-09-15T01:22:14.437 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T01:22:14.493 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T01:22:14.493 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-09-15T01:22:14.495 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-09-15T01:22:14.497 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2024-09-15T01:22:14.500 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-09-15T01:22:14.584 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T01:22:14.680 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T01:22:14.696 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T01:22:14.696 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T01:22:14.697 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T01:22:14.697 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T01:22:14.698 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T01:22:14.698 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T01:22:14.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:22:14.716 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-09-15T01:22:14.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:22:14.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:22:14.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:22:14.718 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-09-15T01:22:14.718 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-15T01:22:14.719 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2024-09-15T01:22:14.774 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-503.el9.x86_64 2024-09-15T01:22:14.774 INFO:teuthology.task.kernel:Running kernel on smithi042: 5.14.0-503.el9.x86_64 2024-09-15T01:22:14.774 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum install -y kernel 2024-09-15T01:22:14.775 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-503.el9.x86_64 2024-09-15T01:22:14.775 INFO:teuthology.task.kernel:Running kernel on smithi175: 5.14.0-503.el9.x86_64 2024-09-15T01:22:14.776 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum install -y kernel 2024-09-15T01:22:14.777 INFO:teuthology.orchestra.run.smithi152.stdout:5.14.0-503.el9.x86_64 2024-09-15T01:22:14.778 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-503.el9.x86_64 2024-09-15T01:22:14.778 INFO:teuthology.task.kernel:Running kernel on smithi152: 5.14.0-503.el9.x86_64 2024-09-15T01:22:14.778 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum install -y kernel 2024-09-15T01:22:14.778 INFO:teuthology.task.kernel:Running kernel on smithi115: 5.14.0-503.el9.x86_64 2024-09-15T01:22:14.778 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum install -y kernel 2024-09-15T01:22:17.179 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-15T01:22:18.093 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.2 MB 00:01 2024-09-15T01:22:18.341 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.2 MB 00:01 2024-09-15T01:22:18.585 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-09-15T01:22:19.363 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-15T01:22:19.927 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-15T01:22:20.158 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-15T01:22:20.445 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-15T01:22:23.922 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.5 MB 00:00 2024-09-15T01:22:24.493 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-15T01:22:25.104 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 6.5 MB 00:00 2024-09-15T01:22:25.347 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 19 kB 00:00 2024-09-15T01:22:25.678 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-15T01:22:25.977 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 19 kB 00:00 2024-09-15T01:22:26.251 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-15T01:22:26.747 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-15T01:22:26.838 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-15T01:22:27.087 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 110 kB/s | 19 kB 00:00 2024-09-15T01:22:27.750 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-15T01:22:28.041 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-15T01:22:31.623 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-15T01:22:32.402 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-15T01:22:33.215 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-15T01:22:33.621 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T01:22:33.621 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T01:22:33.689 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-09-15T01:22:33.704 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:22:33.708 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:22:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-15T01:22:33.708 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:22:33.709 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:22:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T01:22:33.709 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-15T01:22:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T01:22:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T01:22:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T01:22:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:22:33.709 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:22:33.709 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:22:33.709 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-09-15T01:22:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:22:33.710 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 79 M 2024-09-15T01:22:33.710 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 126 M 2024-09-15T01:22:33.710 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:22:33.991 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 181 kB/s | 23 kB 00:00 2024-09-15T01:22:34.392 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T01:22:34.392 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T01:22:34.481 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:22:34.490 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:22:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-15T01:22:34.490 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:22:34.490 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:22:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T01:22:34.490 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-15T01:22:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T01:22:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T01:22:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T01:22:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:22:34.491 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:22:34.491 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:22:34.491 INFO:teuthology.orchestra.run.smithi175.stdout:Install 4 Packages 2024-09-15T01:22:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:22:34.491 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 79 M 2024-09-15T01:22:34.492 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 126 M 2024-09-15T01:22:34.492 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:22:34.887 INFO:teuthology.orchestra.run.smithi175.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 65 kB/s | 23 kB 00:00 2024-09-15T01:22:35.000 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-09-15T01:22:35.222 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T01:22:35.222 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T01:22:35.319 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:22:35.325 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:22:35.325 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-09-15T01:22:35.325 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:22:35.325 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:22:35.325 INFO:teuthology.orchestra.run.smithi152.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T01:22:35.325 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-09-15T01:22:35.325 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T01:22:35.325 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T01:22:35.326 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T01:22:35.326 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:22:35.326 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:22:35.326 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:22:35.326 INFO:teuthology.orchestra.run.smithi152.stdout:Install 4 Packages 2024-09-15T01:22:35.326 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:22:35.327 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 79 M 2024-09-15T01:22:35.327 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 126 M 2024-09-15T01:22:35.327 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:22:35.541 INFO:teuthology.orchestra.run.smithi152.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 198 kB/s | 23 kB 00:00 2024-09-15T01:22:35.675 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 17 MB/s | 29 MB 00:01 2024-09-15T01:22:35.787 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T01:22:35.787 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T01:22:35.882 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:22:35.894 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:22:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-15T01:22:35.894 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:22:35.894 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:22:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T01:22:35.894 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-15T01:22:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T01:22:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T01:22:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T01:22:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:22:35.895 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:22:35.895 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:22:35.895 INFO:teuthology.orchestra.run.smithi042.stdout:Install 4 Packages 2024-09-15T01:22:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:22:35.895 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 79 M 2024-09-15T01:22:35.895 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 126 M 2024-09-15T01:22:35.895 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:22:36.000 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-15T01:22:36.001 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:22:36.001 INFO:teuthology.orchestra.run.smithi115.stdout:Total 34 MB/s | 79 MB 00:02 2024-09-15T01:22:36.146 INFO:teuthology.orchestra.run.smithi175.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 9.7 MB/s | 16 MB 00:01 2024-09-15T01:22:36.182 INFO:teuthology.orchestra.run.smithi042.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 239 kB/s | 23 kB 00:00 2024-09-15T01:22:36.679 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:22:37.024 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:22:37.024 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:22:37.362 INFO:teuthology.orchestra.run.smithi175.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 12 MB/s | 35 MB 00:02 2024-09-15T01:22:37.574 INFO:teuthology.orchestra.run.smithi042.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 21 MB/s | 29 MB 00:01 2024-09-15T01:22:37.979 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:22:37.979 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:22:38.016 INFO:teuthology.orchestra.run.smithi042.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 8.1 MB/s | 16 MB 00:01 2024-09-15T01:22:38.396 INFO:teuthology.orchestra.run.smithi175.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 8.1 MB/s | 29 MB 00:03 2024-09-15T01:22:38.396 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:22:38.396 INFO:teuthology.orchestra.run.smithi175.stdout:Total 20 MB/s | 79 MB 00:03 2024-09-15T01:22:39.084 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:22:39.421 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:22:39.422 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:22:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 9.6 MB/s | 35 MB 00:03 2024-09-15T01:22:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:22:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:Total 21 MB/s | 79 MB 00:03 2024-09-15T01:22:39.916 INFO:teuthology.orchestra.run.smithi152.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 3.5 MB/s | 16 MB 00:04 2024-09-15T01:22:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:22:40.393 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:22:40.401 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:22:40.401 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:22:40.787 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:22:40.788 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:22:41.073 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T01:22:41.159 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:22:41.789 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:22:41.790 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:22:42.124 INFO:teuthology.orchestra.run.smithi152.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 4.3 MB/s | 29 MB 00:06 2024-09-15T01:22:42.858 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:22:43.600 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T01:22:43.666 INFO:teuthology.orchestra.run.smithi152.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 4.2 MB/s | 35 MB 00:08 2024-09-15T01:22:43.666 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:22:43.666 INFO:teuthology.orchestra.run.smithi152.stdout:Total 9.4 MB/s | 79 MB 00:08 2024-09-15T01:22:43.678 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:22:43.946 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:22:44.071 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:22:44.355 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:22:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:22:44.697 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:22:44.697 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:22:45.051 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T01:22:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:22:45.656 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:22:45.657 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:22:46.468 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:22:46.598 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:22:47.941 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:22:48.111 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:22:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:22:48.799 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T01:22:48.893 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:22:48.900 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:22:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:22:51.434 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:22:51.482 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:22:51.723 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:22:51.856 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:22:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:22:53.289 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:22:53.934 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:22:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:22:56.710 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:22:56.747 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:22:58.419 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:01.610 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:32.582 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:32.582 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T01:23:32.582 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:23:32.583 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:23:33.387 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:33.387 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:23:33.387 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:23:33.387 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:33.387 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T01:23:33.387 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T01:23:33.387 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T01:23:33.387 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:23:33.388 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:23:33.746 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T01:23:33.824 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:33.824 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T01:23:33.825 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T01:23:33.825 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T01:23:33.825 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T01:23:33.825 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-15T01:23:33.825 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T01:23:33.825 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T01:23:33.884 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:33.884 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T01:23:33.884 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -qi grub2-tools 2024-09-15T01:23:33.921 INFO:teuthology.orchestra.run.smithi115.stdout:Name : grub2-tools 2024-09-15T01:23:33.921 INFO:teuthology.orchestra.run.smithi115.stdout:Epoch : 1 2024-09-15T01:23:33.921 INFO:teuthology.orchestra.run.smithi115.stdout:Version : 2.06 2024-09-15T01:23:33.921 INFO:teuthology.orchestra.run.smithi115.stdout:Release : 82.el9 2024-09-15T01:23:33.922 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: x86_64 2024-09-15T01:23:33.922 INFO:teuthology.orchestra.run.smithi115.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T01:23:33.922 INFO:teuthology.orchestra.run.smithi115.stdout:Group : Unspecified 2024-09-15T01:23:33.922 INFO:teuthology.orchestra.run.smithi115.stdout:Size : 8274670 2024-09-15T01:23:33.922 INFO:teuthology.orchestra.run.smithi115.stdout:License : GPLv3+ 2024-09-15T01:23:33.922 INFO:teuthology.orchestra.run.smithi115.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T01:23:33.922 INFO:teuthology.orchestra.run.smithi115.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T01:23:33.922 INFO:teuthology.orchestra.run.smithi115.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout:Packager : builder@centos.org 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout:Vendor : CentOS 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout:Summary : Support tools for GRUB. 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout:Description : 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout:hardware devices. 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:23:33.923 INFO:teuthology.orchestra.run.smithi115.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T01:23:33.924 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T01:23:33.925 INFO:teuthology.task.kernel:Updating grub on smithi115 to boot 5.14.0-505.el9.x86_64 2024-09-15T01:23:33.925 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T01:23:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2024-09-15T01:23:34.610 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:34.650 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T01:23:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:23:35.037 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:23:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:23:35.920 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:23:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T01:23:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T01:23:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T01:23:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:23:35.920 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:23:35.921 INFO:teuthology.orchestra.run.smithi115.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T01:23:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:done 2024-09-15T01:23:35.954 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-15T01:23:35.970 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.q0GfX1mkjB 2024-09-15T01:23:35.971 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.q0GfX1mkjB 2024-09-15T01:23:36.075 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.q0GfX1mkjB 2024-09-15T01:23:36.173 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.q0GfX1mkjB is 6KB 2024-09-15T01:23:36.221 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.q0GfX1mkjB 2024-09-15T01:23:36.235 DEBUG:teuthology.orchestra.run.smithi115:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T01:23:36.297 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T01:23:36.300 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T01:23:36.300 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T01:23:36.300 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T01:23:36.300 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T01:23:36.300 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T01:23:36.379 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:36.379 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T01:23:36.379 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T01:23:36.380 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T01:23:36.380 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T01:23:36.380 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-09-15T01:23:36.380 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T01:23:36.380 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T01:23:36.440 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:36.440 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T01:23:36.441 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2024-09-15T01:23:36.457 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2024-09-15T01:23:36.473 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2024-09-15T01:23:36.476 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2024-09-15T01:23:36.476 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.06 2024-09-15T01:23:36.476 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 82.el9 2024-09-15T01:23:36.476 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2024-09-15T01:23:36.476 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T01:23:36.476 INFO:teuthology.orchestra.run.smithi175.stdout:Group : Unspecified 2024-09-15T01:23:36.476 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 8274670 2024-09-15T01:23:36.476 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2024-09-15T01:23:36.476 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T01:23:36.477 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T01:23:36.477 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T01:23:36.477 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T01:23:36.477 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : builder@centos.org 2024-09-15T01:23:36.477 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2024-09-15T01:23:36.477 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T01:23:36.477 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2024-09-15T01:23:36.477 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2024-09-15T01:23:36.477 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:23:36.477 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T01:23:36.477 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T01:23:36.477 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T01:23:36.479 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2024-09-15T01:23:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:23:36.479 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T01:23:36.480 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T01:23:36.480 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 5.14.0-505.el9.x86_64 2024-09-15T01:23:36.480 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T01:23:37.149 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2024-09-15T01:23:37.610 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T01:23:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:23:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:23:38.544 INFO:teuthology.orchestra.run.smithi175.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T01:23:38.583 INFO:teuthology.orchestra.run.smithi175.stderr:done 2024-09-15T01:23:38.586 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-09-15T01:23:38.602 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.0WKrs8r1sd 2024-09-15T01:23:38.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0WKrs8r1sd 2024-09-15T01:23:38.679 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.0WKrs8r1sd 2024-09-15T01:23:38.826 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.0WKrs8r1sd is 6KB 2024-09-15T01:23:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:23:38.827 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:23:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T01:23:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T01:23:38.828 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T01:23:38.828 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:23:38.828 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:23:38.875 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.0WKrs8r1sd 2024-09-15T01:23:38.890 DEBUG:teuthology.orchestra.run.smithi175:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T01:23:38.953 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T01:23:38.953 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T01:23:38.954 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T01:23:38.954 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T01:23:38.955 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T01:23:39.115 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2024-09-15T01:23:39.175 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T01:23:39.255 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:39.256 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T01:23:39.256 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T01:23:39.256 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T01:23:39.256 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T01:23:39.256 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T01:23:39.256 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi042.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T01:23:39.256 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T01:23:39.315 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:39.316 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T01:23:39.316 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -qi grub2-tools 2024-09-15T01:23:39.351 INFO:teuthology.orchestra.run.smithi042.stdout:Name : grub2-tools 2024-09-15T01:23:39.351 INFO:teuthology.orchestra.run.smithi042.stdout:Epoch : 1 2024-09-15T01:23:39.351 INFO:teuthology.orchestra.run.smithi042.stdout:Version : 2.06 2024-09-15T01:23:39.351 INFO:teuthology.orchestra.run.smithi042.stdout:Release : 82.el9 2024-09-15T01:23:39.351 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: x86_64 2024-09-15T01:23:39.351 INFO:teuthology.orchestra.run.smithi042.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T01:23:39.351 INFO:teuthology.orchestra.run.smithi042.stdout:Group : Unspecified 2024-09-15T01:23:39.351 INFO:teuthology.orchestra.run.smithi042.stdout:Size : 8274670 2024-09-15T01:23:39.351 INFO:teuthology.orchestra.run.smithi042.stdout:License : GPLv3+ 2024-09-15T01:23:39.351 INFO:teuthology.orchestra.run.smithi042.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T01:23:39.352 INFO:teuthology.orchestra.run.smithi042.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T01:23:39.352 INFO:teuthology.orchestra.run.smithi042.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T01:23:39.352 INFO:teuthology.orchestra.run.smithi042.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T01:23:39.352 INFO:teuthology.orchestra.run.smithi042.stdout:Packager : builder@centos.org 2024-09-15T01:23:39.352 INFO:teuthology.orchestra.run.smithi042.stdout:Vendor : CentOS 2024-09-15T01:23:39.352 INFO:teuthology.orchestra.run.smithi042.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T01:23:39.352 INFO:teuthology.orchestra.run.smithi042.stdout:Summary : Support tools for GRUB. 2024-09-15T01:23:39.352 INFO:teuthology.orchestra.run.smithi042.stdout:Description : 2024-09-15T01:23:39.352 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:23:39.352 INFO:teuthology.orchestra.run.smithi042.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T01:23:39.352 INFO:teuthology.orchestra.run.smithi042.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T01:23:39.353 INFO:teuthology.orchestra.run.smithi042.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T01:23:39.353 INFO:teuthology.orchestra.run.smithi042.stdout:hardware devices. 2024-09-15T01:23:39.353 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:23:39.353 INFO:teuthology.orchestra.run.smithi042.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T01:23:39.354 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T01:23:39.354 INFO:teuthology.task.kernel:Updating grub on smithi042 to boot 5.14.0-505.el9.x86_64 2024-09-15T01:23:39.354 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T01:23:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-09-15T01:23:40.193 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:40.233 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T01:23:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:23:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:23:41.288 INFO:teuthology.orchestra.run.smithi042.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T01:23:41.317 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-09-15T01:23:41.319 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-09-15T01:23:41.335 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.zIAn6EBnrr 2024-09-15T01:23:41.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zIAn6EBnrr 2024-09-15T01:23:41.366 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:41.367 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:23:41.367 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:23:41.367 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:41.367 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T01:23:41.367 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T01:23:41.367 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T01:23:41.367 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:23:41.367 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:23:41.407 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.zIAn6EBnrr 2024-09-15T01:23:41.551 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.zIAn6EBnrr is 6KB 2024-09-15T01:23:41.600 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.zIAn6EBnrr 2024-09-15T01:23:41.615 DEBUG:teuthology.orchestra.run.smithi042:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T01:23:41.677 INFO:teuthology.orchestra.run.smithi042.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T01:23:41.678 INFO:teuthology.orchestra.run.smithi042.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T01:23:41.678 INFO:teuthology.orchestra.run.smithi042.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T01:23:41.678 INFO:teuthology.orchestra.run.smithi042.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T01:23:41.679 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T01:23:41.741 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T01:23:41.825 DEBUG:teuthology.orchestra.run.smithi042:> sudo shutdown -r now 2024-09-15T01:23:41.873 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:41.874 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T01:23:41.874 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T01:23:41.874 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T01:23:41.874 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T01:23:41.874 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-15T01:23:41.874 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi152.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T01:23:41.875 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T01:23:41.932 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:41.933 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T01:23:41.933 DEBUG:teuthology.orchestra.run.smithi152:> sudo rpm -qi grub2-tools 2024-09-15T01:23:41.974 INFO:teuthology.orchestra.run.smithi152.stdout:Name : grub2-tools 2024-09-15T01:23:41.974 INFO:teuthology.orchestra.run.smithi152.stdout:Epoch : 1 2024-09-15T01:23:41.974 INFO:teuthology.orchestra.run.smithi152.stdout:Version : 2.06 2024-09-15T01:23:41.974 INFO:teuthology.orchestra.run.smithi152.stdout:Release : 82.el9 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:Architecture: x86_64 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:Group : Unspecified 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:Size : 8274670 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:License : GPLv3+ 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:Packager : builder@centos.org 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:Vendor : CentOS 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T01:23:41.975 INFO:teuthology.orchestra.run.smithi152.stdout:Summary : Support tools for GRUB. 2024-09-15T01:23:41.976 INFO:teuthology.orchestra.run.smithi152.stdout:Description : 2024-09-15T01:23:41.976 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:23:41.976 INFO:teuthology.orchestra.run.smithi152.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T01:23:41.976 INFO:teuthology.orchestra.run.smithi152.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T01:23:41.976 INFO:teuthology.orchestra.run.smithi152.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T01:23:41.976 INFO:teuthology.orchestra.run.smithi152.stdout:hardware devices. 2024-09-15T01:23:41.976 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:23:41.976 INFO:teuthology.orchestra.run.smithi152.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T01:23:41.977 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T01:23:41.977 INFO:teuthology.task.kernel:Updating grub on smithi152 to boot 5.14.0-505.el9.x86_64 2024-09-15T01:23:41.977 DEBUG:teuthology.orchestra.run.smithi152:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T01:23:42.705 INFO:teuthology.orchestra.run.smithi152.stderr:Generating grub configuration file ... 2024-09-15T01:23:44.096 INFO:teuthology.orchestra.run.smithi152.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T01:23:44.138 INFO:teuthology.orchestra.run.smithi152.stderr:done 2024-09-15T01:23:44.140 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2024-09-15T01:23:44.155 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.0gqGlRQ96z 2024-09-15T01:23:44.156 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0gqGlRQ96z 2024-09-15T01:23:44.222 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.0gqGlRQ96z 2024-09-15T01:23:44.360 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.0gqGlRQ96z is 6KB 2024-09-15T01:23:44.381 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.0gqGlRQ96z 2024-09-15T01:23:44.396 DEBUG:teuthology.orchestra.run.smithi152:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T01:23:44.459 INFO:teuthology.orchestra.run.smithi152.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T01:23:44.459 INFO:teuthology.orchestra.run.smithi152.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T01:23:44.459 INFO:teuthology.orchestra.run.smithi152.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T01:23:44.459 INFO:teuthology.orchestra.run.smithi152.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T01:23:44.461 DEBUG:teuthology.orchestra.run.smithi152:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T01:23:44.618 DEBUG:teuthology.orchestra.run.smithi152:> sudo shutdown -r now 2024-09-15T01:24:06.481 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-15T01:24:06.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-15T01:24:06.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:24:09.120 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-09-15T01:24:09.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-15T01:24:09.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:24:11.831 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T01:24:11.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-15T01:24:11.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:24:14.621 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-15T01:24:14.621 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-15T01:24:14.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:24:41.149 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-15T01:24:49.089 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-09-15T01:24:50.151 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-15T01:24:50.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:24:58.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-15T01:24:58.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:25:08.541 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-15T01:25:09.124 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T01:25:11.837 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T01:25:16.481 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-09-15T01:25:18.127 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-15T01:25:18.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:25:18.550 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-09-15T01:25:18.900 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-15T01:25:18.901 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-505.el9.x86_64"... 2024-09-15T01:25:18.901 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-09-15T01:25:18.920 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-505.el9.x86_64 2024-09-15T01:25:18.921 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T01:25:18.921 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T01:25:18.921 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T01:25:19.922 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-09-15T01:25:19.922 DEBUG:teuthology.orchestra.run.smithi175:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T01:25:20.028 INFO:teuthology.orchestra.run.smithi175.stdout:ttyS1 2024-09-15T01:25:20.050 DEBUG:teuthology.parallel:result is None 2024-09-15T01:25:20.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-15T01:25:20.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:25:20.837 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-15T01:25:20.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:25:20.954 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-15T01:25:21.093 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-09-15T01:25:21.341 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-15T01:25:21.341 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-15T01:25:21.342 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-15T01:25:21.356 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-505.el9.x86_64 2024-09-15T01:25:21.356 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T01:25:21.356 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T01:25:21.357 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T01:25:21.472 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-15T01:25:21.472 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-15T01:25:21.472 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-09-15T01:25:21.487 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-505.el9.x86_64 2024-09-15T01:25:21.487 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T01:25:21.488 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T01:25:21.488 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T01:25:22.357 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-15T01:25:22.358 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T01:25:22.428 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-09-15T01:25:22.465 DEBUG:teuthology.parallel:result is None 2024-09-15T01:25:22.489 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T01:25:22.489 DEBUG:teuthology.orchestra.run.smithi042:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T01:25:22.573 INFO:teuthology.orchestra.run.smithi042.stdout:ttyS1 2024-09-15T01:25:22.603 DEBUG:teuthology.parallel:result is None 2024-09-15T01:25:28.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-15T01:25:28.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:25:28.818 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-09-15T01:25:29.240 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-15T01:25:29.240 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-505.el9.x86_64"... 2024-09-15T01:25:29.241 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2024-09-15T01:25:29.255 INFO:teuthology.orchestra.run.smithi152.stdout:5.14.0-505.el9.x86_64 2024-09-15T01:25:29.255 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T01:25:29.255 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T01:25:29.255 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T01:25:30.257 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-15T01:25:30.257 DEBUG:teuthology.orchestra.run.smithi152:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T01:25:30.353 INFO:teuthology.orchestra.run.smithi152.stdout:ttyS1 2024-09-15T01:25:30.380 DEBUG:teuthology.parallel:result is None 2024-09-15T01:25:30.381 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T01:25:30.388 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T01:25:30.389 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T01:25:30.392 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T01:25:30.395 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T01:25:30.397 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T01:25:30.418 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T01:25:30.441 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T01:25:30.451 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T01:25:30.451 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T01:25:30.454 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T01:25:30.457 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T01:25:30.461 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T01:25:30.564 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T01:25:30.572 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T01:25:30.573 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T01:25:30.575 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T01:25:30.577 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T01:25:30.580 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T01:25:30.604 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:25:30.609 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:25:30.609 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:25:30.610 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:25:30.617 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:25:30.618 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:25:30.619 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:25:30.622 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:25:30.624 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T01:25:30.631 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T01:25:30.632 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T01:25:30.662 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T01:25:30.664 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T01:25:30.666 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T01:25:30.693 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T01:25:30.704 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T01:25:30.704 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T01:25:30.733 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T01:25:30.735 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T01:25:30.737 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T01:25:30.753 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:25:30.833 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:25:30.881 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:25:30.965 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:25:31.010 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:25:31.010 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T01:25:31.078 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:25:31.123 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:25:31.171 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:25:31.255 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:25:31.297 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:25:31.297 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T01:25:31.363 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:25:31.405 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:25:31.448 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:25:31.528 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:25:31.563 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:25:31.563 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T01:25:31.626 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:25:31.677 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:25:31.726 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:25:31.808 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:25:31.854 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:25:31.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T01:25:31.921 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-09-15T01:25:31.924 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-09-15T01:25:31.926 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2024-09-15T01:25:31.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-09-15T01:25:31.975 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T01:25:31.977 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T01:25:31.984 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T01:25:32.018 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T01:25:32.468 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T01:25:32.476 INFO:teuthology.task.internal:Starting timer... 2024-09-15T01:25:32.477 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T01:25:32.499 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T01:25:32.512 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2024-09-15T01:25:32.575 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2024-09-15T01:25:32.577 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-09-15T01:25:32.662 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-09-15T01:25:32.665 DEBUG:teuthology.orchestra.run.smithi152:> sudo service auditd rotate 2024-09-15T01:25:32.730 INFO:teuthology.orchestra.run.smithi152.stdout:Rotating logs: 2024-09-15T01:25:32.733 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-09-15T01:25:32.800 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-09-15T01:25:32.802 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T01:25:32.802 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2024-09-15T01:25:32.832 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2024-09-15T01:25:32.833 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-09-15T01:25:32.859 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-09-15T01:25:32.859 DEBUG:teuthology.orchestra.run.smithi152:> /usr/sbin/getenforce 2024-09-15T01:25:32.887 INFO:teuthology.orchestra.run.smithi152.stdout:Permissive 2024-09-15T01:25:32.888 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-09-15T01:25:32.928 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-09-15T01:25:32.929 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi042.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi115.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi152.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-09-15T01:25:32.929 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T01:25:32.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:25:32.960 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T01:25:32.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:25:32.990 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T01:25:33.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:25:33.016 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T01:25:33.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:25:33.044 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T01:25:33.044 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2024-09-15T01:25:33.081 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-09-15T01:25:33.110 DEBUG:teuthology.orchestra.run.smithi152:> sudo /usr/sbin/setenforce permissive 2024-09-15T01:25:33.165 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-09-15T01:25:33.202 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T01:25:33.213 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T01:25:33.228 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T01:25:33.229 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi115.front.sepia.ceph.com,smithi152.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T01:29:21.256 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi152.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-09-15T01:29:21.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-15T01:29:21.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:29:21.337 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-09-15T01:29:21.417 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-15T01:29:21.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-15T01:29:21.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:29:21.495 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-15T01:29:21.573 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-15T01:29:21.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-15T01:29:21.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:29:21.647 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-09-15T01:29:21.730 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-15T01:29:21.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-15T01:29:21.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:29:21.809 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-09-15T01:29:21.890 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-15T01:29:21.891 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T01:29:21.902 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T01:29:21.902 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T01:29:21.903 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:29:21.905 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T01:29:21.906 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:29:21.908 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T01:29:21.908 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:29:21.911 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T01:29:21.911 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:29:21.937 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T01:29:21.938 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T01:29:21.942 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T01:29:21.953 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T01:29:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T01:29:21.958 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T01:29:21.963 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T01:29:21.978 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-09-15T01:29:21.979 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T01:29:21.983 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2024-09-15T01:29:21.984 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: ntpd: command not found 2024-09-15T01:29:21.990 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-09-15T01:29:21.995 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2024-09-15T01:29:21.996 INFO:teuthology.orchestra.run.smithi152.stdout:506 Cannot talk to daemon 2024-09-15T01:29:22.005 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-09-15T01:29:22.005 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T01:29:22.011 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T01:29:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T01:29:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-09-15T01:29:22.019 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T01:29:22.025 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T01:29:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T01:29:22.031 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T01:29:22.045 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T01:29:22.079 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-15T01:29:22.082 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T01:29:22.082 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-15T01:29:22.082 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.082 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.083 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.083 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.089 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-09-15T01:29:22.091 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-09-15T01:29:22.093 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T01:29:22.093 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-09-15T01:29:22.093 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.093 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.093 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.093 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.095 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T01:29:22.095 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-09-15T01:29:22.095 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.095 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.095 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.095 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.105 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-09-15T01:29:22.109 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T01:29:22.109 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-09-15T01:29:22.109 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.109 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.109 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.109 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:22.110 INFO:teuthology.run_tasks:Running task install... 2024-09-15T01:29:22.118 DEBUG:teuthology.task.install:project ceph 2024-09-15T01:29:22.118 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-15T01:29:22.118 DEBUG:teuthology.task.install:config {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'} 2024-09-15T01:29:22.118 INFO:teuthology.task.install:Using flavor: default 2024-09-15T01:29:22.123 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T01:29:22.123 INFO:teuthology.task.install:extra packages: [] 2024-09-15T01:29:22.124 DEBUG:teuthology.orchestra.run.smithi042:> sudo dnf -y install 'dnf-command(copr)' 2024-09-15T01:29:22.124 DEBUG:teuthology.orchestra.run.smithi115:> sudo dnf -y install 'dnf-command(copr)' 2024-09-15T01:29:22.125 DEBUG:teuthology.orchestra.run.smithi152:> sudo dnf -y install 'dnf-command(copr)' 2024-09-15T01:29:22.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo dnf -y install 'dnf-command(copr)' 2024-09-15T01:29:22.679 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:48 ago on Sun 15 Sep 2024 01:28:34 AM UTC. 2024-09-15T01:29:22.690 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:48 ago on Sun 15 Sep 2024 01:28:34 AM UTC. 2024-09-15T01:29:22.709 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:47 ago on Sun 15 Sep 2024 01:28:35 AM UTC. 2024-09-15T01:29:22.714 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:45 ago on Sun 15 Sep 2024 01:28:37 AM UTC. 2024-09-15T01:29:22.768 INFO:teuthology.orchestra.run.smithi115.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-15T01:29:22.776 INFO:teuthology.orchestra.run.smithi152.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-15T01:29:22.797 INFO:teuthology.orchestra.run.smithi175.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-15T01:29:22.805 INFO:teuthology.orchestra.run.smithi042.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-15T01:29:22.809 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:29:22.810 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-15T01:29:22.811 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:29:22.818 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:29:22.819 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-09-15T01:29:22.819 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:29:22.839 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:29:22.841 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-15T01:29:22.841 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:29:22.849 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:29:22.851 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-15T01:29:22.851 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:29:22.879 DEBUG:teuthology.orchestra.run.smithi115:> sudo dnf -y copr enable ceph/el9 2024-09-15T01:29:22.884 DEBUG:teuthology.orchestra.run.smithi152:> sudo dnf -y copr enable ceph/el9 2024-09-15T01:29:22.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo dnf -y copr enable ceph/el9 2024-09-15T01:29:22.912 DEBUG:teuthology.orchestra.run.smithi042:> sudo dnf -y copr enable ceph/el9 2024-09-15T01:29:23.048 INFO:teuthology.orchestra.run.smithi115.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-15T01:29:23.048 INFO:teuthology.orchestra.run.smithi115.stderr:of the main distribution, and quality may vary. 2024-09-15T01:29:23.048 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-15T01:29:23.048 INFO:teuthology.orchestra.run.smithi115.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-15T01:29:23.049 INFO:teuthology.orchestra.run.smithi115.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-15T01:29:23.049 INFO:teuthology.orchestra.run.smithi115.stderr:, 2024-09-15T01:29:23.049 INFO:teuthology.orchestra.run.smithi115.stderr:and packages are not held to any quality or security level. 2024-09-15T01:29:23.049 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-15T01:29:23.049 INFO:teuthology.orchestra.run.smithi115.stderr:Please do not file bug reports about these packages in Fedora 2024-09-15T01:29:23.049 INFO:teuthology.orchestra.run.smithi115.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-15T01:29:23.056 INFO:teuthology.orchestra.run.smithi152.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-15T01:29:23.056 INFO:teuthology.orchestra.run.smithi152.stderr:of the main distribution, and quality may vary. 2024-09-15T01:29:23.056 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-09-15T01:29:23.057 INFO:teuthology.orchestra.run.smithi152.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-15T01:29:23.057 INFO:teuthology.orchestra.run.smithi152.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-15T01:29:23.057 INFO:teuthology.orchestra.run.smithi152.stderr:, 2024-09-15T01:29:23.057 INFO:teuthology.orchestra.run.smithi152.stderr:and packages are not held to any quality or security level. 2024-09-15T01:29:23.057 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-09-15T01:29:23.057 INFO:teuthology.orchestra.run.smithi152.stderr:Please do not file bug reports about these packages in Fedora 2024-09-15T01:29:23.057 INFO:teuthology.orchestra.run.smithi152.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-15T01:29:23.072 INFO:teuthology.orchestra.run.smithi175.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-15T01:29:23.072 INFO:teuthology.orchestra.run.smithi175.stderr:of the main distribution, and quality may vary. 2024-09-15T01:29:23.073 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-09-15T01:29:23.073 INFO:teuthology.orchestra.run.smithi175.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-15T01:29:23.073 INFO:teuthology.orchestra.run.smithi175.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-15T01:29:23.073 INFO:teuthology.orchestra.run.smithi175.stderr:, 2024-09-15T01:29:23.073 INFO:teuthology.orchestra.run.smithi175.stderr:and packages are not held to any quality or security level. 2024-09-15T01:29:23.073 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-09-15T01:29:23.073 INFO:teuthology.orchestra.run.smithi175.stderr:Please do not file bug reports about these packages in Fedora 2024-09-15T01:29:23.073 INFO:teuthology.orchestra.run.smithi175.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-15T01:29:23.100 INFO:teuthology.orchestra.run.smithi042.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-15T01:29:23.100 INFO:teuthology.orchestra.run.smithi042.stderr:of the main distribution, and quality may vary. 2024-09-15T01:29:23.100 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T01:29:23.100 INFO:teuthology.orchestra.run.smithi042.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-15T01:29:23.100 INFO:teuthology.orchestra.run.smithi042.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-15T01:29:23.100 INFO:teuthology.orchestra.run.smithi042.stderr:, 2024-09-15T01:29:23.100 INFO:teuthology.orchestra.run.smithi042.stderr:and packages are not held to any quality or security level. 2024-09-15T01:29:23.101 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T01:29:23.101 INFO:teuthology.orchestra.run.smithi042.stderr:Please do not file bug reports about these packages in Fedora 2024-09-15T01:29:23.101 INFO:teuthology.orchestra.run.smithi042.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-15T01:29:23.104 INFO:teuthology.orchestra.run.smithi115.stdout:Repository successfully enabled. 2024-09-15T01:29:23.113 INFO:teuthology.orchestra.run.smithi152.stdout:Repository successfully enabled. 2024-09-15T01:29:23.124 INFO:teuthology.orchestra.run.smithi175.stdout:Repository successfully enabled. 2024-09-15T01:29:23.138 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'tag': None, 'wait_for_package': False} 2024-09-15T01:29:23.139 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:29:23.151 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'tag': None, 'wait_for_package': False} 2024-09-15T01:29:23.151 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:29:23.156 INFO:teuthology.orchestra.run.smithi042.stdout:Repository successfully enabled. 2024-09-15T01:29:23.158 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'tag': None, 'wait_for_package': False} 2024-09-15T01:29:23.159 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:29:23.192 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'tag': None, 'wait_for_package': False} 2024-09-15T01:29:23.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:29:23.299 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/ 2024-09-15T01:29:23.299 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1643.gd75f2382 2024-09-15T01:29:23.303 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/ 2024-09-15T01:29:23.303 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1643.gd75f2382 2024-09-15T01:29:23.336 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/ 2024-09-15T01:29:23.336 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1643.gd75f2382 2024-09-15T01:29:23.345 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/ 2024-09-15T01:29:23.345 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1643.gd75f2382 2024-09-15T01:29:23.503 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T01:29:23.503 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:29:23.503 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T01:29:23.507 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T01:29:23.507 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:29:23.507 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T01:29:23.524 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T01:29:23.524 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:29:23.524 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T01:29:23.526 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T01:29:23.526 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:29:23.526 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T01:29:23.530 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T01:29:23.530 DEBUG:teuthology.orchestra.run.smithi152:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d75f23820806da622f561ee5ee7551015285993f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T01:29:23.532 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T01:29:23.532 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d75f23820806da622f561ee5ee7551015285993f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T01:29:23.551 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T01:29:23.551 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d75f23820806da622f561ee5ee7551015285993f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T01:29:23.552 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T01:29:23.552 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d75f23820806da622f561ee5ee7551015285993f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T01:29:23.597 DEBUG:teuthology.orchestra.run.smithi152:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T01:29:23.598 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T01:29:23.617 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T01:29:23.619 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T01:29:23.671 DEBUG:teuthology.orchestra.run.smithi152:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T01:29:23.673 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T01:29:23.692 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T01:29:23.696 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T01:29:23.736 INFO:teuthology.orchestra.run.smithi152.stdout:check_obsoletes = 1 2024-09-15T01:29:23.737 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum clean all 2024-09-15T01:29:23.738 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-09-15T01:29:23.739 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-09-15T01:29:23.757 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-09-15T01:29:23.758 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-09-15T01:29:23.762 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2024-09-15T01:29:23.764 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2024-09-15T01:29:23.964 INFO:teuthology.orchestra.run.smithi115.stdout:45 files removed 2024-09-15T01:29:23.978 INFO:teuthology.orchestra.run.smithi152.stdout:45 files removed 2024-09-15T01:29:23.988 INFO:teuthology.orchestra.run.smithi175.stdout:45 files removed 2024-09-15T01:29:23.991 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw 2024-09-15T01:29:23.993 INFO:teuthology.orchestra.run.smithi042.stdout:45 files removed 2024-09-15T01:29:24.005 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw 2024-09-15T01:29:24.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw 2024-09-15T01:29:24.021 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw 2024-09-15T01:29:24.968 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 112 kB/s | 77 kB 00:00 2024-09-15T01:29:24.973 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 115 kB/s | 77 kB 00:00 2024-09-15T01:29:24.978 INFO:teuthology.orchestra.run.smithi152.stdout:ceph packages for x86_64 113 kB/s | 77 kB 00:00 2024-09-15T01:29:24.983 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 117 kB/s | 77 kB 00:00 2024-09-15T01:29:25.476 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-09-15T01:29:25.482 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-09-15T01:29:25.495 INFO:teuthology.orchestra.run.smithi152.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-09-15T01:29:25.533 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 21 kB/s | 11 kB 00:00 2024-09-15T01:29:25.893 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 4.7 kB/s | 1.8 kB 00:00 2024-09-15T01:29:25.896 INFO:teuthology.orchestra.run.smithi152.stdout:ceph source packages 4.9 kB/s | 1.8 kB 00:00 2024-09-15T01:29:25.900 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 4.7 kB/s | 1.8 kB 00:00 2024-09-15T01:29:25.926 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 5.1 kB/s | 1.8 kB 00:00 2024-09-15T01:29:26.044 INFO:teuthology.orchestra.run.smithi115.stdout:Copr repo for el9 owned by ceph 66 kB/s | 7.6 kB 00:00 2024-09-15T01:29:26.063 INFO:teuthology.orchestra.run.smithi152.stdout:Copr repo for el9 owned by ceph 58 kB/s | 7.6 kB 00:00 2024-09-15T01:29:26.066 INFO:teuthology.orchestra.run.smithi175.stdout:Copr repo for el9 owned by ceph 58 kB/s | 7.6 kB 00:00 2024-09-15T01:29:26.084 INFO:teuthology.orchestra.run.smithi042.stdout:Copr repo for el9 owned by ceph 63 kB/s | 7.6 kB 00:00 2024-09-15T01:29:26.477 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.2 MB 00:00 2024-09-15T01:29:27.693 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.2 MB 00:01 2024-09-15T01:29:27.747 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-15T01:29:28.027 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.2 MB 00:01 2024-09-15T01:29:29.723 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-15T01:29:29.870 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 20 MB 00:02 2024-09-15T01:29:30.928 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 20 MB 00:02 2024-09-15T01:29:33.990 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-15T01:29:34.245 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-15T01:29:35.544 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 20 MB 00:06 2024-09-15T01:29:35.796 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-15T01:29:36.378 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-15T01:29:36.416 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-15T01:29:36.683 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-15T01:29:37.353 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-15T01:29:37.904 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 19 kB 00:00 2024-09-15T01:29:39.016 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-15T01:29:40.278 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.5 MB 00:00 2024-09-15T01:29:41.945 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-15T01:29:42.188 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-15T01:29:42.757 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-15T01:29:42.783 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-15T01:29:44.556 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-15T01:29:44.656 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-15T01:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 14 M 2024-09-15T01:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-09-15T01:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.1 M 2024-09-15T01:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-15T01:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-15T01:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 5.0 M 2024-09-15T01:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 22 M 2024-09-15T01:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 637 k 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 458 k 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 4.9 M 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 45 k 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 120 k 2024-09-15T01:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 156 k 2024-09-15T01:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T01:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 320 k 2024-09-15T01:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 295 k 2024-09-15T01:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 100 k 2024-09-15T01:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T01:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout:Install 22 Packages 2024-09-15T01:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 2 Packages 2024-09-15T01:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 58 M 2024-09-15T01:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:29:45.266 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:29:45.281 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:29:45.281 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 14 M 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrading: 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.1 M 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 5.0 M 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 22 M 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 637 k 2024-09-15T01:29:45.282 INFO:teuthology.orchestra.run.smithi152.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 458 k 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 4.9 M 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 45 k 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 120 k 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 156 k 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 320 k 2024-09-15T01:29:45.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 295 k 2024-09-15T01:29:45.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 100 k 2024-09-15T01:29:45.284 INFO:teuthology.orchestra.run.smithi152.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T01:29:45.284 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:29:45.284 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:29:45.284 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:29:45.284 INFO:teuthology.orchestra.run.smithi152.stdout:Install 22 Packages 2024-09-15T01:29:45.284 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrade 2 Packages 2024-09-15T01:29:45.284 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:29:45.285 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 58 M 2024-09-15T01:29:45.285 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:29:46.075 INFO:teuthology.orchestra.run.smithi115.stdout:(1/24): ceph-base-17.2.7-1643.gd75f2382.el9.x86 6.3 MB/s | 5.0 MB 00:00 2024-09-15T01:29:46.108 INFO:teuthology.orchestra.run.smithi115.stdout:(2/24): ceph-selinux-17.2.7-1643.gd75f2382.el9. 762 kB/s | 25 kB 00:00 2024-09-15T01:29:46.258 INFO:teuthology.orchestra.run.smithi115.stdout:(3/24): libcephfs2-17.2.7-1643.gd75f2382.el9.x8 4.2 MB/s | 637 kB 00:00 2024-09-15T01:29:46.359 INFO:teuthology.orchestra.run.smithi115.stdout:(4/24): libradosstriper1-17.2.7-1643.gd75f2382. 4.5 MB/s | 458 kB 00:00 2024-09-15T01:29:46.501 INFO:teuthology.orchestra.run.smithi115.stdout:(5/24): ceph-radosgw-17.2.7-1643.gd75f2382.el9. 12 MB/s | 14 MB 00:01 2024-09-15T01:29:46.568 INFO:teuthology.orchestra.run.smithi115.stdout:(6/24): python3-ceph-argparse-17.2.7-1643.gd75f 679 kB/s | 45 kB 00:00 2024-09-15T01:29:46.659 INFO:teuthology.orchestra.run.smithi115.stdout:(7/24): python3-ceph-common-17.2.7-1643.gd75f23 1.3 MB/s | 120 kB 00:00 2024-09-15T01:29:46.743 INFO:teuthology.orchestra.run.smithi115.stdout:(8/24): librgw2-17.2.7-1643.gd75f2382.el9.x86_6 13 MB/s | 4.9 MB 00:00 2024-09-15T01:29:46.768 INFO:teuthology.orchestra.run.smithi115.stdout:(9/24): python3-cephfs-17.2.7-1643.gd75f2382.el 1.4 MB/s | 156 kB 00:00 2024-09-15T01:29:46.794 INFO:teuthology.orchestra.run.smithi115.stdout:(10/24): python3-rados-17.2.7-1643.gd75f2382.el 6.2 MB/s | 320 kB 00:00 2024-09-15T01:29:46.819 INFO:teuthology.orchestra.run.smithi115.stdout:(11/24): python3-rbd-17.2.7-1643.gd75f2382.el9. 5.7 MB/s | 295 kB 00:00 2024-09-15T01:29:46.852 INFO:teuthology.orchestra.run.smithi115.stdout:(12/24): python3-rgw-17.2.7-1643.gd75f2382.el9. 1.7 MB/s | 100 kB 00:00 2024-09-15T01:29:46.884 INFO:teuthology.orchestra.run.smithi152.stdout:(1/24): ceph-base-17.2.7-1643.gd75f2382.el9.x86 3.5 MB/s | 5.0 MB 00:01 2024-09-15T01:29:46.935 INFO:teuthology.orchestra.run.smithi152.stdout:(2/24): ceph-selinux-17.2.7-1643.gd75f2382.el9. 504 kB/s | 25 kB 00:00 2024-09-15T01:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout:(3/24): libcephfs2-17.2.7-1643.gd75f2382.el9.x8 6.8 MB/s | 637 kB 00:00 2024-09-15T01:29:47.110 INFO:teuthology.orchestra.run.smithi152.stdout:(4/24): libradosstriper1-17.2.7-1643.gd75f2382. 5.4 MB/s | 458 kB 00:00 2024-09-15T01:29:47.111 INFO:teuthology.orchestra.run.smithi115.stdout:(13/24): ceph-common-17.2.7-1643.gd75f2382.el9. 12 MB/s | 22 MB 00:01 2024-09-15T01:29:47.117 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:29:47.131 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 14 M 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.1 M 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 5.0 M 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 22 M 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T01:29:47.132 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 637 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 458 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 4.9 M 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 45 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 120 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 156 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 320 k 2024-09-15T01:29:47.133 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 295 k 2024-09-15T01:29:47.134 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 100 k 2024-09-15T01:29:47.134 INFO:teuthology.orchestra.run.smithi042.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T01:29:47.134 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:29:47.134 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:29:47.134 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:29:47.134 INFO:teuthology.orchestra.run.smithi042.stdout:Install 22 Packages 2024-09-15T01:29:47.134 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 2 Packages 2024-09-15T01:29:47.134 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:29:47.135 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 58 M 2024-09-15T01:29:47.135 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:29:47.211 INFO:teuthology.orchestra.run.smithi115.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 126 kB/s | 45 kB 00:00 2024-09-15T01:29:47.237 INFO:teuthology.orchestra.run.smithi115.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 384 kB/s | 160 kB 00:00 2024-09-15T01:29:47.304 INFO:teuthology.orchestra.run.smithi115.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-15T01:29:47.329 INFO:teuthology.orchestra.run.smithi115.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 460 kB/s | 42 kB 00:00 2024-09-15T01:29:47.354 INFO:teuthology.orchestra.run.smithi115.stdout:(18/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-15T01:29:47.402 INFO:teuthology.orchestra.run.smithi152.stdout:(5/24): ceph-radosgw-17.2.7-1643.gd75f2382.el9. 7.4 MB/s | 14 MB 00:01 2024-09-15T01:29:47.404 INFO:teuthology.orchestra.run.smithi115.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2024-09-15T01:29:47.430 INFO:teuthology.orchestra.run.smithi115.stdout:(20/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.4 MB/s | 308 kB 00:00 2024-09-15T01:29:47.455 INFO:teuthology.orchestra.run.smithi115.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 788 kB/s | 67 kB 00:00 2024-09-15T01:29:47.461 INFO:teuthology.orchestra.run.smithi152.stdout:(6/24): python3-ceph-argparse-17.2.7-1643.gd75f 778 kB/s | 45 kB 00:00 2024-09-15T01:29:47.505 INFO:teuthology.orchestra.run.smithi115.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-15T01:29:47.536 INFO:teuthology.orchestra.run.smithi152.stdout:(7/24): python3-ceph-common-17.2.7-1643.gd75f23 1.6 MB/s | 120 kB 00:00 2024-09-15T01:29:47.670 INFO:teuthology.orchestra.run.smithi152.stdout:(8/24): librgw2-17.2.7-1643.gd75f2382.el9.x86_6 8.8 MB/s | 4.9 MB 00:00 2024-09-15T01:29:47.695 INFO:teuthology.orchestra.run.smithi152.stdout:(9/24): python3-cephfs-17.2.7-1643.gd75f2382.el 983 kB/s | 156 kB 00:00 2024-09-15T01:29:47.739 INFO:teuthology.orchestra.run.smithi115.stdout:(23/24): librbd1-17.2.7-1643.gd75f2382.el9.x86_ 10 MB/s | 2.9 MB 00:00 2024-09-15T01:29:47.798 INFO:teuthology.orchestra.run.smithi115.stdout:(24/24): librados2-17.2.7-1643.gd75f2382.el9.x8 8.4 MB/s | 3.1 MB 00:00 2024-09-15T01:29:47.803 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:29:47.803 INFO:teuthology.orchestra.run.smithi115.stdout:Total 19 MB/s | 58 MB 00:03 2024-09-15T01:29:47.865 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:29:47.898 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:29:47.898 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:29:47.937 INFO:teuthology.orchestra.run.smithi152.stdout:(10/24): ceph-common-17.2.7-1643.gd75f2382.el9. 9.0 MB/s | 22 MB 00:02 2024-09-15T01:29:47.962 INFO:teuthology.orchestra.run.smithi152.stdout:(11/24): python3-rados-17.2.7-1643.gd75f2382.el 1.1 MB/s | 320 kB 00:00 2024-09-15T01:29:47.979 INFO:teuthology.orchestra.run.smithi152.stdout:(12/24): python3-rgw-17.2.7-1643.gd75f2382.el9. 2.3 MB/s | 100 kB 00:00 2024-09-15T01:29:48.004 INFO:teuthology.orchestra.run.smithi152.stdout:(13/24): python3-rbd-17.2.7-1643.gd75f2382.el9. 954 kB/s | 295 kB 00:00 2024-09-15T01:29:48.088 INFO:teuthology.orchestra.run.smithi152.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-09-15T01:29:48.130 INFO:teuthology.orchestra.run.smithi152.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 959 kB/s | 160 kB 00:00 2024-09-15T01:29:48.172 INFO:teuthology.orchestra.run.smithi152.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-15T01:29:48.188 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-15T01:29:48.205 INFO:teuthology.orchestra.run.smithi152.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-15T01:29:48.231 INFO:teuthology.orchestra.run.smithi152.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-09-15T01:29:48.272 INFO:teuthology.orchestra.run.smithi152.stdout:(19/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-15T01:29:48.290 INFO:teuthology.orchestra.run.smithi042.stdout:(1/24): ceph-base-17.2.7-1643.gd75f2382.el9.x86 4.8 MB/s | 5.0 MB 00:01 2024-09-15T01:29:48.298 INFO:teuthology.orchestra.run.smithi152.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 537 kB/s | 49 kB 00:00 2024-09-15T01:29:48.315 INFO:teuthology.orchestra.run.smithi152.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 879 kB/s | 67 kB 00:00 2024-09-15T01:29:48.349 INFO:teuthology.orchestra.run.smithi042.stdout:(2/24): ceph-selinux-17.2.7-1643.gd75f2382.el9. 432 kB/s | 25 kB 00:00 2024-09-15T01:29:48.356 INFO:teuthology.orchestra.run.smithi152.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-15T01:29:48.507 INFO:teuthology.orchestra.run.smithi152.stdout:(23/24): librbd1-17.2.7-1643.gd75f2382.el9.x86_ 15 MB/s | 2.9 MB 00:00 2024-09-15T01:29:48.553 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:29:48.553 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:29:48.566 INFO:teuthology.orchestra.run.smithi042.stdout:(3/24): ceph-common-17.2.7-1643.gd75f2382.el9.x 17 MB/s | 22 MB 00:01 2024-09-15T01:29:48.598 INFO:teuthology.orchestra.run.smithi152.stdout:(24/24): librados2-17.2.7-1643.gd75f2382.el9.x8 10 MB/s | 3.1 MB 00:00 2024-09-15T01:29:48.603 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:29:48.604 INFO:teuthology.orchestra.run.smithi152.stdout:Total 18 MB/s | 58 MB 00:03 2024-09-15T01:29:48.665 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:29:48.699 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:29:48.699 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:29:48.724 INFO:teuthology.orchestra.run.smithi042.stdout:(4/24): libcephfs2-17.2.7-1643.gd75f2382.el9.x8 1.7 MB/s | 637 kB 00:00 2024-09-15T01:29:48.758 INFO:teuthology.orchestra.run.smithi042.stdout:(5/24): libradosstriper1-17.2.7-1643.gd75f2382. 2.3 MB/s | 458 kB 00:00 2024-09-15T01:29:48.817 INFO:teuthology.orchestra.run.smithi042.stdout:(6/24): python3-ceph-argparse-17.2.7-1643.gd75f 775 kB/s | 45 kB 00:00 2024-09-15T01:29:48.967 INFO:teuthology.orchestra.run.smithi042.stdout:(7/24): ceph-radosgw-17.2.7-1643.gd75f2382.el9. 8.5 MB/s | 14 MB 00:01 2024-09-15T01:29:48.992 INFO:teuthology.orchestra.run.smithi042.stdout:(8/24): python3-ceph-common-17.2.7-1643.gd75f23 686 kB/s | 120 kB 00:00 2024-09-15T01:29:49.017 INFO:teuthology.orchestra.run.smithi042.stdout:(9/24): python3-cephfs-17.2.7-1643.gd75f2382.el 3.1 MB/s | 156 kB 00:00 2024-09-15T01:29:49.043 INFO:teuthology.orchestra.run.smithi042.stdout:(10/24): python3-rados-17.2.7-1643.gd75f2382.el 6.2 MB/s | 320 kB 00:00 2024-09-15T01:29:49.076 INFO:teuthology.orchestra.run.smithi042.stdout:(11/24): python3-rbd-17.2.7-1643.gd75f2382.el9. 4.9 MB/s | 295 kB 00:00 2024-09-15T01:29:49.102 INFO:teuthology.orchestra.run.smithi042.stdout:(12/24): python3-rgw-17.2.7-1643.gd75f2382.el9. 1.7 MB/s | 100 kB 00:00 2024-09-15T01:29:49.185 INFO:teuthology.orchestra.run.smithi042.stdout:(13/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-15T01:29:49.211 INFO:teuthology.orchestra.run.smithi042.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-15T01:29:49.278 INFO:teuthology.orchestra.run.smithi042.stdout:(15/24): librgw2-17.2.7-1643.gd75f2382.el9.x86_ 8.9 MB/s | 4.9 MB 00:00 2024-09-15T01:29:49.328 INFO:teuthology.orchestra.run.smithi042.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-15T01:29:49.338 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:29:49.338 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:29:49.361 INFO:teuthology.orchestra.run.smithi042.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-09-15T01:29:49.404 INFO:teuthology.orchestra.run.smithi042.stdout:(18/24): python3-prettytable-0.7.2-27.el9.noarc 336 kB/s | 42 kB 00:00 2024-09-15T01:29:49.437 INFO:teuthology.orchestra.run.smithi042.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 647 kB/s | 49 kB 00:00 2024-09-15T01:29:49.471 INFO:teuthology.orchestra.run.smithi042.stdout:(20/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.1 MB/s | 308 kB 00:00 2024-09-15T01:29:49.496 INFO:teuthology.orchestra.run.smithi042.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 750 kB/s | 67 kB 00:00 2024-09-15T01:29:49.555 INFO:teuthology.orchestra.run.smithi042.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-15T01:29:49.688 INFO:teuthology.orchestra.run.smithi042.stdout:(23/24): librbd1-17.2.7-1643.gd75f2382.el9.x86_ 15 MB/s | 2.9 MB 00:00 2024-09-15T01:29:49.763 INFO:teuthology.orchestra.run.smithi042.stdout:(24/24): librados2-17.2.7-1643.gd75f2382.el9.x8 11 MB/s | 3.1 MB 00:00 2024-09-15T01:29:49.769 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:29:49.769 INFO:teuthology.orchestra.run.smithi042.stdout:Total 22 MB/s | 58 MB 00:02 2024-09-15T01:29:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:29:49.831 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:29:49.869 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:29:49.869 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:29:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-15T01:29:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-15T01:29:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:50.473 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:50.518 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-15T01:29:50.546 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:29:50.547 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-15T01:29:50.577 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:29:50.578 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:29:50.688 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:29:50.699 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:29:50.699 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-15T01:29:50.699 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:29:50.699 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:29:50.699 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 14 M 2024-09-15T01:29:50.699 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-09-15T01:29:50.700 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.1 M 2024-09-15T01:29:50.700 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-15T01:29:50.700 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-15T01:29:50.700 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 5.0 M 2024-09-15T01:29:50.700 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 22 M 2024-09-15T01:29:50.700 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-15T01:29:50.700 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T01:29:50.700 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 637 k 2024-09-15T01:29:50.700 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T01:29:50.700 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T01:29:50.700 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T01:29:50.700 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 458 k 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 4.9 M 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 45 k 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 120 k 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 156 k 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 320 k 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 295 k 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 100 k 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:29:50.701 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:29:50.702 INFO:teuthology.orchestra.run.smithi175.stdout:Install 22 Packages 2024-09-15T01:29:50.702 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 2 Packages 2024-09-15T01:29:50.702 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:29:50.702 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 58 M 2024-09-15T01:29:50.702 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:29:50.886 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-15T01:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:50.986 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-15T01:29:51.012 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-15T01:29:51.058 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-15T01:29:51.076 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:51.094 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-15T01:29:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 10/26 2024-09-15T01:29:51.162 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-15T01:29:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-15T01:29:51.193 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-15T01:29:51.256 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-15T01:29:51.282 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:51.322 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-15T01:29:51.329 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-15T01:29:51.365 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-15T01:29:51.371 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-15T01:29:51.406 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-15T01:29:51.448 INFO:teuthology.orchestra.run.smithi175.stdout:(1/24): ceph-base-17.2.7-1643.gd75f2382.el9.x86 9.3 MB/s | 5.0 MB 00:00 2024-09-15T01:29:51.448 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-15T01:29:51.515 INFO:teuthology.orchestra.run.smithi175.stdout:(2/24): ceph-selinux-17.2.7-1643.gd75f2382.el9. 378 kB/s | 25 kB 00:00 2024-09-15T01:29:51.623 INFO:teuthology.orchestra.run.smithi175.stdout:(3/24): libcephfs2-17.2.7-1643.gd75f2382.el9.x8 5.8 MB/s | 637 kB 00:00 2024-09-15T01:29:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-15T01:29:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-15T01:29:51.798 INFO:teuthology.orchestra.run.smithi175.stdout:(4/24): ceph-radosgw-17.2.7-1643.gd75f2382.el9. 16 MB/s | 14 MB 00:00 2024-09-15T01:29:51.802 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-15T01:29:51.823 INFO:teuthology.orchestra.run.smithi175.stdout:(5/24): libradosstriper1-17.2.7-1643.gd75f2382. 2.2 MB/s | 458 kB 00:00 2024-09-15T01:29:51.830 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-15T01:29:51.861 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:29:51.876 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-15T01:29:51.876 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-15T01:29:51.890 INFO:teuthology.orchestra.run.smithi175.stdout:(6/24): python3-ceph-argparse-17.2.7-1643.gd75f 676 kB/s | 45 kB 00:00 2024-09-15T01:29:51.912 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-15T01:29:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 19/26 2024-09-15T01:29:51.955 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 10/26 2024-09-15T01:29:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 20/26 2024-09-15T01:29:51.992 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-15T01:29:52.007 INFO:teuthology.orchestra.run.smithi175.stdout:(7/24): librgw2-17.2.7-1643.gd75f2382.el9.x86_6 24 MB/s | 4.9 MB 00:00 2024-09-15T01:29:52.033 INFO:teuthology.orchestra.run.smithi175.stdout:(8/24): python3-ceph-common-17.2.7-1643.gd75f23 848 kB/s | 120 kB 00:00 2024-09-15T01:29:52.054 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-15T01:29:52.058 INFO:teuthology.orchestra.run.smithi175.stdout:(9/24): python3-cephfs-17.2.7-1643.gd75f2382.el 3.0 MB/s | 156 kB 00:00 2024-09-15T01:29:52.082 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-15T01:29:52.083 INFO:teuthology.orchestra.run.smithi175.stdout:(10/24): python3-rados-17.2.7-1643.gd75f2382.el 6.2 MB/s | 320 kB 00:00 2024-09-15T01:29:52.109 INFO:teuthology.orchestra.run.smithi175.stdout:(11/24): python3-rbd-17.2.7-1643.gd75f2382.el9. 5.7 MB/s | 295 kB 00:00 2024-09-15T01:29:52.134 INFO:teuthology.orchestra.run.smithi175.stdout:(12/24): python3-rgw-17.2.7-1643.gd75f2382.el9. 1.9 MB/s | 100 kB 00:00 2024-09-15T01:29:52.148 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-15T01:29:52.198 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-15T01:29:52.232 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-15T01:29:52.232 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-15T01:29:52.265 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-15T01:29:52.332 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:52.351 INFO:teuthology.orchestra.run.smithi175.stdout:(13/24): ceph-common-17.2.7-1643.gd75f2382.el9. 16 MB/s | 22 MB 00:01 2024-09-15T01:29:52.409 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:52.489 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-15T01:29:52.517 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:52.558 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-15T01:29:52.589 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:52.594 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-15T01:29:52.637 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-15T01:29:52.667 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-15T01:29:52.680 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-15T01:29:52.741 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 19/26 2024-09-15T01:29:52.771 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 20/26 2024-09-15T01:29:52.859 INFO:teuthology.orchestra.run.smithi175.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 62 kB/s | 45 kB 00:00 2024-09-15T01:29:52.993 INFO:teuthology.orchestra.run.smithi175.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 181 kB/s | 160 kB 00:00 2024-09-15T01:29:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-15T01:29:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-15T01:29:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-15T01:29:53.295 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-15T01:29:53.310 INFO:teuthology.orchestra.run.smithi175.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 691 kB/s | 662 kB 00:00 2024-09-15T01:29:53.338 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 10/26 2024-09-15T01:29:53.360 INFO:teuthology.orchestra.run.smithi175.stdout:(17/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 6.1 MB/s | 308 kB 00:00 2024-09-15T01:29:53.375 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-15T01:29:53.385 INFO:teuthology.orchestra.run.smithi175.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-15T01:29:53.419 INFO:teuthology.orchestra.run.smithi175.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-15T01:29:53.439 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-15T01:29:53.464 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-15T01:29:53.477 INFO:teuthology.orchestra.run.smithi175.stdout:(20/24): thrift-0.15.0-2.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-09-15T01:29:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-15T01:29:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-15T01:29:53.595 INFO:teuthology.orchestra.run.smithi175.stdout:(21/24): python3-prettytable-0.7.2-27.el9.noarc 70 kB/s | 42 kB 00:00 2024-09-15T01:29:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-15T01:29:53.645 INFO:teuthology.orchestra.run.smithi175.stdout:(22/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 373 kB/s | 292 kB 00:00 2024-09-15T01:29:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-15T01:29:53.728 INFO:teuthology.orchestra.run.smithi175.stdout:(23/24): librados2-17.2.7-1643.gd75f2382.el9.x8 12 MB/s | 3.1 MB 00:00 2024-09-15T01:29:53.862 INFO:teuthology.orchestra.run.smithi175.stdout:(24/24): librbd1-17.2.7-1643.gd75f2382.el9.x86_ 11 MB/s | 2.9 MB 00:00 2024-09-15T01:29:53.866 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:29:53.867 INFO:teuthology.orchestra.run.smithi175.stdout:Total 18 MB/s | 58 MB 00:03 2024-09-15T01:29:53.929 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:29:53.956 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-15T01:29:53.961 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:29:53.962 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:29:53.993 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-15T01:29:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-15T01:29:54.144 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 19/26 2024-09-15T01:29:54.179 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 20/26 2024-09-15T01:29:54.415 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:54.625 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:29:54.626 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:29:54.994 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:55.029 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-15T01:29:55.116 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-15T01:29:55.116 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T01:29:55.116 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T01:29:55.116 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:29:55.147 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:29:55.170 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:55.209 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:55.735 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:55.779 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-15T01:29:55.853 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:29:55.858 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-15T01:29:55.858 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T01:29:55.858 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T01:29:55.859 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:29:55.897 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:29:56.243 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-15T01:29:56.298 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:56.476 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-15T01:29:56.499 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:56.588 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:56.635 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-15T01:29:56.670 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-15T01:29:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:57.111 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-15T01:29:57.135 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-15T01:29:57.198 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-15T01:29:57.259 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-15T01:29:57.303 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 10/26 2024-09-15T01:29:57.340 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-15T01:29:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:57.387 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-15T01:29:57.409 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-15T01:29:57.437 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-15T01:29:57.495 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-15T01:29:57.495 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T01:29:57.496 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T01:29:57.496 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:29:57.503 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-15T01:29:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:29:57.556 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-15T01:29:57.587 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-15T01:29:57.620 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-15T01:29:57.922 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-15T01:29:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-15T01:29:58.048 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-15T01:29:58.124 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 19/26 2024-09-15T01:29:58.152 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 20/26 2024-09-15T01:30:00.516 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:30:00.577 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:30:01.099 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:30:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-15T01:30:01.236 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-15T01:30:01.236 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T01:30:01.236 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T01:30:01.236 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:01.270 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:30:05.134 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:30:05.134 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-09-15T01:30:05.134 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-09-15T01:30:05.135 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-09-15T01:30:05.135 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-09-15T01:30:05.135 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-09-15T01:30:05.135 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-09-15T01:30:05.135 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-09-15T01:30:05.135 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:30:05.987 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /sys 2024-09-15T01:30:05.987 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /proc 2024-09-15T01:30:05.987 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /mnt 2024-09-15T01:30:05.987 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /var/tmp 2024-09-15T01:30:05.987 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /home 2024-09-15T01:30:05.987 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /root 2024-09-15T01:30:05.987 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /tmp 2024-09-15T01:30:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:06.438 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-15T01:30:06.498 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-15T01:30:06.498 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:06.498 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T01:30:06.498 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:30:06.498 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:30:06.498 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:06.500 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T01:30:06.583 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T01:30:06.584 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-15T01:30:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-15T01:30:07.351 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:07.351 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T01:30:07.351 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:30:07.352 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:30:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T01:30:07.436 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T01:30:07.436 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 1/26 2024-09-15T01:30:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:30:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-15T01:30:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:30:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 5/26 2024-09-15T01:30:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 6/26 2024-09-15T01:30:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 7/26 2024-09-15T01:30:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 8/26 2024-09-15T01:30:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 9/26 2024-09-15T01:30:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 10/26 2024-09-15T01:30:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-15T01:30:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-15T01:30:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 13/26 2024-09-15T01:30:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-15T01:30:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-15T01:30:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-15T01:30:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-15T01:30:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-15T01:30:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-15T01:30:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-15T01:30:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-15T01:30:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-15T01:30:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:30:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-15T01:30:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 25/26 2024-09-15T01:30:07.985 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:30:07.985 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2024-09-15T01:30:07.986 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2024-09-15T01:30:07.986 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2024-09-15T01:30:07.986 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2024-09-15T01:30:07.986 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2024-09-15T01:30:07.986 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /root 2024-09-15T01:30:07.986 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2024-09-15T01:30:07.986 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:08.656 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:08.656 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 1/26 2024-09-15T01:30:08.656 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:30:08.656 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-15T01:30:08.656 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:30:08.656 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 5/26 2024-09-15T01:30:08.657 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 6/26 2024-09-15T01:30:08.657 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 7/26 2024-09-15T01:30:08.657 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 8/26 2024-09-15T01:30:08.657 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 9/26 2024-09-15T01:30:08.657 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 10/26 2024-09-15T01:30:08.657 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-15T01:30:08.657 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-15T01:30:08.657 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 13/26 2024-09-15T01:30:08.657 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-15T01:30:08.657 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-15T01:30:08.657 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-15T01:30:08.657 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-15T01:30:08.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-15T01:30:08.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-15T01:30:08.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-15T01:30:08.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-15T01:30:08.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-15T01:30:08.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:30:08.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-15T01:30:08.658 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 25/26 2024-09-15T01:30:08.767 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T01:30:08.768 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T01:30:08.769 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T01:30:08.769 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.769 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T01:30:08.769 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.769 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T01:30:08.769 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T01:30:08.769 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.769 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.769 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.769 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T01:30:08.769 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.769 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.770 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:08.770 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T01:30:08.770 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:08.770 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:30:09.157 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-test 2024-09-15T01:30:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-15T01:30:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-15T01:30:09.436 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:09.436 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T01:30:09.436 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:30:09.436 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:30:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T01:30:09.472 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:09.472 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:09.472 INFO:teuthology.orchestra.run.smithi152.stdout:Upgraded: 2024-09-15T01:30:09.472 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.473 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.473 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:30:09.473 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.473 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.473 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.473 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.473 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T01:30:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.475 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.475 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:09.475 INFO:teuthology.orchestra.run.smithi152.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T01:30:09.475 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:09.475 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:30:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T01:30:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:09.732 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:27 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:09.797 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-test 2024-09-15T01:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-15T01:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 48 M 2024-09-15T01:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-15T01:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 163 k 2024-09-15T01:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T01:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T01:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-09-15T01:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 48 M 2024-09-15T01:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 193 M 2024-09-15T01:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:30:10.157 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.9 MB/s | 304 kB 00:00 2024-09-15T01:30:10.199 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-15T01:30:10.240 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): libcephsqlite-17.2.7-1643.gd75f2382.el9. 675 kB/s | 163 kB 00:00 2024-09-15T01:30:10.365 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:28 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:10.500 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:30:10.501 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:10.501 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-09-15T01:30:10.501 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:10.501 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:30:10.501 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 48 M 2024-09-15T01:30:10.501 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-09-15T01:30:10.501 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 163 k 2024-09-15T01:30:10.501 INFO:teuthology.orchestra.run.smithi152.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T01:30:10.501 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T01:30:10.501 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:10.501 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:30:10.501 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:10.502 INFO:teuthology.orchestra.run.smithi152.stdout:Install 4 Packages 2024-09-15T01:30:10.502 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:10.502 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 48 M 2024-09-15T01:30:10.502 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 193 M 2024-09-15T01:30:10.502 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:30:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 1/26 2024-09-15T01:30:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:30:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-15T01:30:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 5/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 6/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 7/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 8/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 9/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 10/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 13/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-15T01:30:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-15T01:30:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-15T01:30:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-15T01:30:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-15T01:30:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-15T01:30:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-15T01:30:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:30:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-15T01:30:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 25/26 2024-09-15T01:30:11.134 INFO:teuthology.orchestra.run.smithi152.stdout:(1/4): libcephsqlite-17.2.7-1643.gd75f2382.el9. 461 kB/s | 163 kB 00:00 2024-09-15T01:30:11.461 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:30:11.461 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-09-15T01:30:11.461 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-09-15T01:30:11.461 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-09-15T01:30:11.461 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-09-15T01:30:11.462 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-09-15T01:30:11.462 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-09-15T01:30:11.462 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-09-15T01:30:11.462 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:11.482 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): ceph-test-17.2.7-1643.gd75f2382.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-15T01:30:11.484 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:11.485 INFO:teuthology.orchestra.run.smithi115.stdout:Total 30 MB/s | 48 MB 00:01 2024-09-15T01:30:11.512 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:30:11.531 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:30:11.531 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-09-15T01:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T01:30:11.543 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T01:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T01:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T01:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T01:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T01:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T01:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T01:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T01:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:11.545 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:30:11.551 INFO:teuthology.orchestra.run.smithi152.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 395 kB/s | 304 kB 00:00 2024-09-15T01:30:11.743 INFO:teuthology.orchestra.run.smithi152.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 105 kB/s | 64 kB 00:00 2024-09-15T01:30:11.889 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-test 2024-09-15T01:30:11.898 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:30:11.899 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:30:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:30:12.477 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:28 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:30:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T01:30:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T01:30:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-15T01:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:30:12.614 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 48 M 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 163 k 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout:Install 4 Packages 2024-09-15T01:30:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 48 M 2024-09-15T01:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 193 M 2024-09-15T01:30:12.616 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:30:12.810 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-15T01:30:12.848 INFO:teuthology.orchestra.run.smithi042.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-09-15T01:30:12.888 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-15T01:30:12.888 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:12.889 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T01:30:12.889 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:30:12.889 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:30:12.889 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:12.890 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T01:30:12.923 INFO:teuthology.orchestra.run.smithi042.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 850 kB/s | 64 kB 00:00 2024-09-15T01:30:12.971 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T01:30:12.971 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:13.073 INFO:teuthology.orchestra.run.smithi042.stdout:(3/4): libcephsqlite-17.2.7-1643.gd75f2382.el9. 397 kB/s | 163 kB 00:00 2024-09-15T01:30:13.335 INFO:teuthology.orchestra.run.smithi152.stdout:(4/4): ceph-test-17.2.7-1643.gd75f2382.el9.x86_ 19 MB/s | 48 MB 00:02 2024-09-15T01:30:13.337 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:13.337 INFO:teuthology.orchestra.run.smithi152.stdout:Total 17 MB/s | 48 MB 00:02 2024-09-15T01:30:13.365 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:30:13.385 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:30:13.385 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:30:13.751 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:30:13.751 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:30:14.297 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:30:14.362 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:14.363 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 1/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 5/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 6/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 7/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 8/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 9/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 10/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 13/26 2024-09-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 25/26 2024-09-15T01:30:14.369 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T01:30:14.418 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T01:30:14.445 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-15T01:30:15.160 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:15.160 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:15.160 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-09-15T01:30:15.160 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.160 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.160 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:30:15.160 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.160 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.160 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.160 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.161 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T01:30:15.162 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.162 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.162 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:15.162 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T01:30:15.162 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:15.162 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:30:15.540 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-test 2024-09-15T01:30:16.107 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:28 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:30:16.247 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:30:16.248 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:16.248 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-15T01:30:16.248 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:16.248 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:30:16.248 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 48 M 2024-09-15T01:30:16.248 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-15T01:30:16.248 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 163 k 2024-09-15T01:30:16.248 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T01:30:16.249 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T01:30:16.249 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:16.249 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:30:16.249 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:16.249 INFO:teuthology.orchestra.run.smithi175.stdout:Install 4 Packages 2024-09-15T01:30:16.249 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:16.249 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 48 M 2024-09-15T01:30:16.250 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 193 M 2024-09-15T01:30:16.250 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:30:16.265 INFO:teuthology.orchestra.run.smithi042.stdout:(4/4): ceph-test-17.2.7-1643.gd75f2382.el9.x86_ 13 MB/s | 48 MB 00:03 2024-09-15T01:30:16.267 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:16.267 INFO:teuthology.orchestra.run.smithi042.stdout:Total 13 MB/s | 48 MB 00:03 2024-09-15T01:30:16.296 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:30:16.317 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:30:16.317 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:30:16.703 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:30:16.703 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:30:16.709 INFO:teuthology.orchestra.run.smithi175.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-15T01:30:16.759 INFO:teuthology.orchestra.run.smithi175.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-15T01:30:16.859 INFO:teuthology.orchestra.run.smithi175.stdout:(3/4): libcephsqlite-17.2.7-1643.gd75f2382.el9. 459 kB/s | 163 kB 00:00 2024-09-15T01:30:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-15T01:30:17.114 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-15T01:30:17.271 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:30:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T01:30:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T01:30:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-15T01:30:18.925 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-15T01:30:19.009 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-15T01:30:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-15T01:30:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 1/4 2024-09-15T01:30:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2/4 2024-09-15T01:30:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T01:30:19.531 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T01:30:19.531 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:19.531 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:30:19.531 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:19.531 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:19.531 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T01:30:19.531 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T01:30:19.531 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:19.531 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:30:19.781 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph 2024-09-15T01:30:20.345 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:38 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:20.489 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:30:20.491 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:20.491 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-15T01:30:20.491 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:20.491 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:30:20.491 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 6.5 k 2024-09-15T01:30:20.491 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-15T01:30:20.491 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.1 M 2024-09-15T01:30:20.491 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 1.4 M 2024-09-15T01:30:20.491 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 240 k 2024-09-15T01:30:20.492 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.9 M 2024-09-15T01:30:20.492 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 15 M 2024-09-15T01:30:20.492 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T01:30:20.492 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T01:30:20.492 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T01:30:20.492 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T01:30:20.492 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T01:30:20.492 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T01:30:20.492 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T01:30:20.492 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections 2024-09-15T01:30:20.492 INFO:teuthology.orchestra.run.smithi115.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T01:30:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T01:30:20.494 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T01:30:20.494 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T01:30:20.494 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T01:30:20.494 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T01:30:20.494 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T01:30:20.494 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T01:30:20.494 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T01:30:20.494 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:20.494 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:30:20.494 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:20.494 INFO:teuthology.orchestra.run.smithi115.stdout:Install 34 Packages 2024-09-15T01:30:20.494 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:20.495 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 27 M 2024-09-15T01:30:20.495 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 96 M 2024-09-15T01:30:20.495 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:30:20.518 INFO:teuthology.orchestra.run.smithi175.stdout:(4/4): ceph-test-17.2.7-1643.gd75f2382.el9.x86_ 12 MB/s | 48 MB 00:04 2024-09-15T01:30:20.520 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:20.520 INFO:teuthology.orchestra.run.smithi175.stdout:Total 11 MB/s | 48 MB 00:04 2024-09-15T01:30:20.547 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:30:20.567 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:30:20.567 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:30:20.938 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:30:20.939 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:30:21.057 INFO:teuthology.orchestra.run.smithi115.stdout:(1/34): ceph-17.2.7-1643.gd75f2382.el9.x86_64.r 37 kB/s | 6.5 kB 00:00 2024-09-15T01:30:21.186 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-15T01:30:21.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 1/4 2024-09-15T01:30:21.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2/4 2024-09-15T01:30:21.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T01:30:21.461 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:30:21.490 INFO:teuthology.orchestra.run.smithi115.stdout:(2/34): ceph-mds-17.2.7-1643.gd75f2382.el9.x86_ 3.4 MB/s | 2.1 MB 00:00 2024-09-15T01:30:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T01:30:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:21.522 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:30:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T01:30:21.522 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T01:30:21.523 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:21.523 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:30:21.524 INFO:teuthology.orchestra.run.smithi115.stdout:(3/34): ceph-mgr-17.2.7-1643.gd75f2382.el9.x86_ 2.2 MB/s | 1.4 MB 00:00 2024-09-15T01:30:21.541 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T01:30:21.583 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T01:30:21.599 INFO:teuthology.orchestra.run.smithi115.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1643.gd75f 3.1 MB/s | 240 kB 00:00 2024-09-15T01:30:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-15T01:30:21.792 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph 2024-09-15T01:30:21.799 INFO:teuthology.orchestra.run.smithi115.stdout:(5/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-15T01:30:21.925 INFO:teuthology.orchestra.run.smithi115.stdout:(6/34): python3-cryptography-36.0.1-4.el9.x86_6 9.8 MB/s | 1.2 MB 00:00 2024-09-15T01:30:21.975 INFO:teuthology.orchestra.run.smithi115.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-15T01:30:22.025 INFO:teuthology.orchestra.run.smithi115.stdout:(8/34): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-09-15T01:30:22.076 INFO:teuthology.orchestra.run.smithi115.stdout:(9/34): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-15T01:30:22.082 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-15T01:30:22.151 INFO:teuthology.orchestra.run.smithi115.stdout:(10/34): ceph-mon-17.2.7-1643.gd75f2382.el9.x86 3.6 MB/s | 3.9 MB 00:01 2024-09-15T01:30:22.159 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-15T01:30:22.176 INFO:teuthology.orchestra.run.smithi115.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 2.1 MB/s | 215 kB 00:00 2024-09-15T01:30:22.366 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:40 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:22.435 INFO:teuthology.orchestra.run.smithi115.stdout:(12/34): python3-markupsafe-1.1.1-12.el9.x86_64 134 kB/s | 35 kB 00:00 2024-09-15T01:30:22.502 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:30:22.504 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:22.504 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-09-15T01:30:22.504 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:22.504 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:30:22.504 INFO:teuthology.orchestra.run.smithi152.stdout: ceph x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 6.5 k 2024-09-15T01:30:22.504 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-09-15T01:30:22.504 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.1 M 2024-09-15T01:30:22.504 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 1.4 M 2024-09-15T01:30:22.505 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 240 k 2024-09-15T01:30:22.505 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.9 M 2024-09-15T01:30:22.505 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 15 M 2024-09-15T01:30:22.505 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T01:30:22.505 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T01:30:22.505 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T01:30:22.505 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T01:30:22.505 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T01:30:22.505 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T01:30:22.505 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T01:30:22.505 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections 2024-09-15T01:30:22.505 INFO:teuthology.orchestra.run.smithi152.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T01:30:22.506 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T01:30:22.507 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T01:30:22.507 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T01:30:22.507 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T01:30:22.507 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T01:30:22.507 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T01:30:22.507 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T01:30:22.507 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:22.507 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:30:22.507 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:22.507 INFO:teuthology.orchestra.run.smithi152.stdout:Install 34 Packages 2024-09-15T01:30:22.507 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:22.508 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 27 M 2024-09-15T01:30:22.508 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 96 M 2024-09-15T01:30:22.508 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:30:22.602 INFO:teuthology.orchestra.run.smithi115.stdout:(13/34): ceph-osd-17.2.7-1643.gd75f2382.el9.x86 14 MB/s | 15 MB 00:01 2024-09-15T01:30:22.627 INFO:teuthology.orchestra.run.smithi115.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 362 kB/s | 172 kB 00:00 2024-09-15T01:30:22.861 INFO:teuthology.orchestra.run.smithi115.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 181 kB/s | 77 kB 00:00 2024-09-15T01:30:22.894 INFO:teuthology.orchestra.run.smithi115.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 156 kB/s | 42 kB 00:00 2024-09-15T01:30:22.928 INFO:teuthology.orchestra.run.smithi115.stdout:(17/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 651 kB/s | 43 kB 00:00 2024-09-15T01:30:22.953 INFO:teuthology.orchestra.run.smithi115.stdout:(18/34): python3-pytz-2021.1-5.el9.noarch.rpm 145 kB/s | 51 kB 00:00 2024-09-15T01:30:22.986 INFO:teuthology.orchestra.run.smithi115.stdout:(19/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.9 MB/s | 172 kB 00:00 2024-09-15T01:30:23.020 INFO:teuthology.orchestra.run.smithi115.stdout:(20/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 160 kB/s | 11 kB 00:00 2024-09-15T01:30:23.046 INFO:teuthology.orchestra.run.smithi115.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 309 kB/s | 18 kB 00:00 2024-09-15T01:30:23.071 INFO:teuthology.orchestra.run.smithi115.stdout:(22/34): python3-cherrypy-18.6.1-2.el9.noarch.r 2.5 MB/s | 358 kB 00:00 2024-09-15T01:30:23.096 INFO:teuthology.orchestra.run.smithi115.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 390 kB/s | 19 kB 00:00 2024-09-15T01:30:23.121 INFO:teuthology.orchestra.run.smithi115.stdout:(24/34): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-09-15T01:30:23.146 INFO:teuthology.orchestra.run.smithi115.stdout:(25/34): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-15T01:30:23.150 INFO:teuthology.orchestra.run.smithi152.stdout:(1/34): ceph-17.2.7-1643.gd75f2382.el9.x86_64.r 58 kB/s | 6.5 kB 00:00 2024-09-15T01:30:23.172 INFO:teuthology.orchestra.run.smithi115.stdout:(26/34): python3-logutils-0.3.5-21.el9.noarch.r 612 kB/s | 46 kB 00:00 2024-09-15T01:30:23.197 INFO:teuthology.orchestra.run.smithi115.stdout:(27/34): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-15T01:30:23.223 INFO:teuthology.orchestra.run.smithi115.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-15T01:30:23.248 INFO:teuthology.orchestra.run.smithi115.stdout:(29/34): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-15T01:30:23.273 INFO:teuthology.orchestra.run.smithi115.stdout:(30/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-15T01:30:23.298 INFO:teuthology.orchestra.run.smithi115.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-15T01:30:23.323 INFO:teuthology.orchestra.run.smithi115.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 802 kB/s | 20 kB 00:00 2024-09-15T01:30:23.349 INFO:teuthology.orchestra.run.smithi115.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-15T01:30:23.351 INFO:teuthology.orchestra.run.smithi152.stdout:(2/34): ceph-mds-17.2.7-1643.gd75f2382.el9.x86_ 6.6 MB/s | 2.1 MB 00:00 2024-09-15T01:30:23.374 INFO:teuthology.orchestra.run.smithi115.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-15T01:30:23.379 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:23.379 INFO:teuthology.orchestra.run.smithi115.stdout:Total 9.4 MB/s | 27 MB 00:02 2024-09-15T01:30:23.393 INFO:teuthology.orchestra.run.smithi152.stdout:(3/34): ceph-mgr-17.2.7-1643.gd75f2382.el9.x86_ 4.0 MB/s | 1.4 MB 00:00 2024-09-15T01:30:23.485 INFO:teuthology.orchestra.run.smithi152.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1643.gd75f 2.6 MB/s | 240 kB 00:00 2024-09-15T01:30:23.504 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:30:23.541 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:30:23.541 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:30:23.669 INFO:teuthology.orchestra.run.smithi152.stdout:(5/34): ceph-mon-17.2.7-1643.gd75f2382.el9.x86_ 7.6 MB/s | 3.9 MB 00:00 2024-09-15T01:30:23.835 INFO:teuthology.orchestra.run.smithi152.stdout:(6/34): ceph-osd-17.2.7-1643.gd75f2382.el9.x86_ 32 MB/s | 15 MB 00:00 2024-09-15T01:30:23.928 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:30:23.928 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:30:24.119 INFO:teuthology.orchestra.run.smithi152.stdout:(7/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 399 kB/s | 253 kB 00:00 2024-09-15T01:30:24.244 INFO:teuthology.orchestra.run.smithi152.stdout:(8/34): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-15T01:30:24.294 INFO:teuthology.orchestra.run.smithi152.stdout:(9/34): python3-ply-3.11-14.el9.noarch.rpm 232 kB/s | 106 kB 00:00 2024-09-15T01:30:24.361 INFO:teuthology.orchestra.run.smithi152.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 1.0 MB/s | 125 kB 00:00 2024-09-15T01:30:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-15T01:30:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 1/4 2024-09-15T01:30:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2/4 2024-09-15T01:30:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T01:30:24.503 INFO:teuthology.orchestra.run.smithi152.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 1.0 MB/s | 215 kB 00:00 2024-09-15T01:30:24.528 INFO:teuthology.orchestra.run.smithi152.stdout:(12/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-15T01:30:24.562 INFO:teuthology.orchestra.run.smithi152.stdout:(13/34): python3-cryptography-36.0.1-4.el9.x86_ 1.4 MB/s | 1.2 MB 00:00 2024-09-15T01:30:24.579 INFO:teuthology.orchestra.run.smithi152.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-15T01:30:24.629 INFO:teuthology.orchestra.run.smithi152.stdout:(15/34): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-09-15T01:30:24.654 INFO:teuthology.orchestra.run.smithi152.stdout:(16/34): python3-markupsafe-1.1.1-12.el9.x86_64 230 kB/s | 35 kB 00:00 2024-09-15T01:30:24.696 INFO:teuthology.orchestra.run.smithi152.stdout:(17/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 650 kB/s | 43 kB 00:00 2024-09-15T01:30:24.722 INFO:teuthology.orchestra.run.smithi152.stdout:(18/34): python3-pytz-2021.1-5.el9.noarch.rpm 320 kB/s | 51 kB 00:00 2024-09-15T01:30:24.739 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T01:30:24.739 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:24.740 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:30:24.740 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:24.740 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:24.740 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T01:30:24.740 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T01:30:24.740 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:24.740 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:30:24.755 INFO:teuthology.orchestra.run.smithi152.stdout:(19/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-15T01:30:24.780 INFO:teuthology.orchestra.run.smithi152.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 758 kB/s | 18 kB 00:00 2024-09-15T01:30:24.806 INFO:teuthology.orchestra.run.smithi152.stdout:(21/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 128 kB/s | 11 kB 00:00 2024-09-15T01:30:24.831 INFO:teuthology.orchestra.run.smithi152.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-15T01:30:24.856 INFO:teuthology.orchestra.run.smithi152.stdout:(23/34): python3-cherrypy-18.6.1-2.el9.noarch.r 2.2 MB/s | 358 kB 00:00 2024-09-15T01:30:24.881 INFO:teuthology.orchestra.run.smithi152.stdout:(24/34): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-09-15T01:30:24.906 INFO:teuthology.orchestra.run.smithi152.stdout:(25/34): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-09-15T01:30:24.932 INFO:teuthology.orchestra.run.smithi152.stdout:(26/34): python3-logutils-0.3.5-21.el9.noarch.r 616 kB/s | 46 kB 00:00 2024-09-15T01:30:24.949 INFO:teuthology.orchestra.run.smithi152.stdout:(27/34): python3-more-itertools-8.12.0-2.el9.no 1.1 MB/s | 79 kB 00:00 2024-09-15T01:30:24.974 INFO:teuthology.orchestra.run.smithi152.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 391 kB/s | 16 kB 00:00 2024-09-15T01:30:24.999 INFO:teuthology.orchestra.run.smithi152.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-15T01:30:25.024 INFO:teuthology.orchestra.run.smithi152.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 716 kB/s | 36 kB 00:00 2024-09-15T01:30:25.025 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph 2024-09-15T01:30:25.049 INFO:teuthology.orchestra.run.smithi152.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-15T01:30:25.075 INFO:teuthology.orchestra.run.smithi152.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 796 kB/s | 20 kB 00:00 2024-09-15T01:30:25.100 INFO:teuthology.orchestra.run.smithi152.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-15T01:30:25.125 INFO:teuthology.orchestra.run.smithi152.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-15T01:30:25.129 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:25.130 INFO:teuthology.orchestra.run.smithi152.stdout:Total 10 MB/s | 27 MB 00:02 2024-09-15T01:30:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:30:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-15T01:30:25.259 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-15T01:30:25.259 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:30:25.292 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-15T01:30:25.297 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:30:25.297 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:30:25.382 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-15T01:30:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-15T01:30:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-15T01:30:25.555 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-15T01:30:25.585 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-15T01:30:25.619 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-15T01:30:25.623 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:41 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:30:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-15T01:30:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-15T01:30:25.710 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:30:25.711 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:30:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-15T01:30:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-15T01:30:25.766 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:30:25.769 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-15T01:30:25.769 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:25.769 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:30:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 6.5 k 2024-09-15T01:30:25.769 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-15T01:30:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.1 M 2024-09-15T01:30:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 1.4 M 2024-09-15T01:30:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 240 k 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.9 M 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 15 M 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T01:30:25.770 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T01:30:25.771 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T01:30:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T01:30:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T01:30:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T01:30:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T01:30:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:25.772 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:30:25.772 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:25.772 INFO:teuthology.orchestra.run.smithi042.stdout:Install 34 Packages 2024-09-15T01:30:25.772 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:25.773 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 27 M 2024-09-15T01:30:25.773 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 96 M 2024-09-15T01:30:25.773 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:30:25.806 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-15T01:30:25.846 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T01:30:25.920 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-15T01:30:26.058 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-15T01:30:26.115 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-15T01:30:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-15T01:30:26.192 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-15T01:30:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-15T01:30:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-15T01:30:26.295 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-15T01:30:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-15T01:30:26.498 INFO:teuthology.orchestra.run.smithi042.stdout:(1/34): ceph-17.2.7-1643.gd75f2382.el9.x86_64.r 51 kB/s | 6.5 kB 00:00 2024-09-15T01:30:26.546 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-15T01:30:26.592 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-15T01:30:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-15T01:30:26.724 INFO:teuthology.orchestra.run.smithi042.stdout:(2/34): ceph-mgr-17.2.7-1643.gd75f2382.el9.x86_ 4.0 MB/s | 1.4 MB 00:00 2024-09-15T01:30:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-15T01:30:26.774 INFO:teuthology.orchestra.run.smithi042.stdout:(3/34): ceph-mds-17.2.7-1643.gd75f2382.el9.x86_ 5.1 MB/s | 2.1 MB 00:00 2024-09-15T01:30:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-15T01:30:26.874 INFO:teuthology.orchestra.run.smithi042.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1643.gd75f 2.3 MB/s | 240 kB 00:00 2024-09-15T01:30:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-15T01:30:26.995 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:30:27.041 INFO:teuthology.orchestra.run.smithi042.stdout:(5/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-15T01:30:27.053 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 29/34 2024-09-15T01:30:27.070 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-15T01:30:27.077 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-15T01:30:27.102 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-15T01:30:27.102 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:27.102 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T01:30:27.102 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T01:30:27.103 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T01:30:27.103 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:27.103 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-15T01:30:27.135 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-15T01:30:27.141 INFO:teuthology.orchestra.run.smithi042.stdout:(6/34): ceph-mon-17.2.7-1643.gd75f2382.el9.x86_ 6.1 MB/s | 3.9 MB 00:00 2024-09-15T01:30:27.227 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-15T01:30:27.275 INFO:teuthology.orchestra.run.smithi042.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 798 kB/s | 106 kB 00:00 2024-09-15T01:30:27.296 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-15T01:30:27.341 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-15T01:30:27.359 INFO:teuthology.orchestra.run.smithi042.stdout:(8/34): python3-pycparser-2.20-6.el9.noarch.rpm 1.6 MB/s | 135 kB 00:00 2024-09-15T01:30:27.384 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-15T01:30:27.414 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-15T01:30:27.417 INFO:teuthology.orchestra.run.smithi042.stdout:(9/34): python3-cryptography-36.0.1-4.el9.x86_6 3.3 MB/s | 1.2 MB 00:00 2024-09-15T01:30:27.448 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-15T01:30:27.481 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-15T01:30:27.493 INFO:teuthology.orchestra.run.smithi042.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 934 kB/s | 125 kB 00:00 2024-09-15T01:30:27.518 INFO:teuthology.orchestra.run.smithi042.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 2.1 MB/s | 215 kB 00:00 2024-09-15T01:30:27.522 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-15T01:30:27.557 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-15T01:30:27.568 INFO:teuthology.orchestra.run.smithi042.stdout:(12/34): python3-markupsafe-1.1.1-12.el9.x86_64 696 kB/s | 35 kB 00:00 2024-09-15T01:30:27.583 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-15T01:30:27.593 INFO:teuthology.orchestra.run.smithi042.stdout:(13/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-15T01:30:27.619 INFO:teuthology.orchestra.run.smithi042.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-15T01:30:27.632 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-15T01:30:27.644 INFO:teuthology.orchestra.run.smithi042.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-15T01:30:27.663 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T01:30:27.678 INFO:teuthology.orchestra.run.smithi042.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-09-15T01:30:27.711 INFO:teuthology.orchestra.run.smithi042.stdout:(17/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 650 kB/s | 43 kB 00:00 2024-09-15T01:30:27.737 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-15T01:30:27.745 INFO:teuthology.orchestra.run.smithi042.stdout:(18/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.6 MB/s | 172 kB 00:00 2024-09-15T01:30:27.779 INFO:teuthology.orchestra.run.smithi042.stdout:(19/34): python3-cherrypy-18.6.1-2.el9.noarch.r 5.4 MB/s | 358 kB 00:00 2024-09-15T01:30:27.804 INFO:teuthology.orchestra.run.smithi042.stdout:(20/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-09-15T01:30:27.829 INFO:teuthology.orchestra.run.smithi042.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-15T01:30:27.855 INFO:teuthology.orchestra.run.smithi042.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-15T01:30:27.880 INFO:teuthology.orchestra.run.smithi042.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-15T01:30:27.885 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-15T01:30:27.905 INFO:teuthology.orchestra.run.smithi042.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-15T01:30:27.930 INFO:teuthology.orchestra.run.smithi042.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 919 kB/s | 46 kB 00:00 2024-09-15T01:30:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-15T01:30:27.979 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-15T01:30:27.981 INFO:teuthology.orchestra.run.smithi042.stdout:(27/34): python3-portend-3.1.0-2.el9.noarch.rpm 659 kB/s | 16 kB 00:00 2024-09-15T01:30:28.006 INFO:teuthology.orchestra.run.smithi042.stdout:(28/34): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-15T01:30:28.031 INFO:teuthology.orchestra.run.smithi042.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-15T01:30:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-15T01:30:28.057 INFO:teuthology.orchestra.run.smithi042.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-15T01:30:28.070 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-15T01:30:28.120 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-15T01:30:28.175 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-15T01:30:28.207 INFO:teuthology.orchestra.run.smithi042.stdout:(31/34): ceph-osd-17.2.7-1643.gd75f2382.el9.x86 10 MB/s | 15 MB 00:01 2024-09-15T01:30:28.233 INFO:teuthology.orchestra.run.smithi042.stdout:(32/34): python3-webob-1.8.8-2.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-09-15T01:30:28.274 INFO:teuthology.orchestra.run.smithi042.stdout:(33/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.9 MB/s | 427 kB 00:00 2024-09-15T01:30:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:(34/34): python3-zc-lockfile-2.0-10.el9.noarch. 218 kB/s | 20 kB 00:00 2024-09-15T01:30:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:Total 11 MB/s | 27 MB 00:02 2024-09-15T01:30:28.372 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-15T01:30:28.425 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-15T01:30:28.436 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:30:28.443 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-15T01:30:28.443 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 1/4 2024-09-15T01:30:28.443 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2/4 2024-09-15T01:30:28.443 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T01:30:28.477 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:30:28.477 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:30:28.481 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-15T01:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-15T01:30:28.510 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-15T01:30:28.510 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:28.510 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T01:30:28.510 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T01:30:28.510 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T01:30:28.510 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:28.592 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-15T01:30:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-15T01:30:28.713 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T01:30:28.713 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:28.713 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:30:28.713 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:28.713 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:28.713 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T01:30:28.713 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T01:30:28.713 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:28.713 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:30:28.748 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-15T01:30:28.886 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:30:28.886 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:30:28.890 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-15T01:30:28.895 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 29/34 2024-09-15T01:30:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-15T01:30:28.914 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:28.914 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T01:30:28.914 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T01:30:28.915 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T01:30:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:28.934 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:29.006 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph 2024-09-15T01:30:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-15T01:30:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-15T01:30:29.148 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:29.148 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T01:30:29.148 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T01:30:29.148 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T01:30:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:29.213 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-15T01:30:29.573 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:41 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:30:29.716 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:30:29.718 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:29.718 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-15T01:30:29.718 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:29.718 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:30:29.718 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 6.5 k 2024-09-15T01:30:29.718 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.1 M 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 1.4 M 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 240 k 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.9 M 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 15 M 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T01:30:29.719 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections 2024-09-15T01:30:29.720 INFO:teuthology.orchestra.run.smithi175.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-15T01:30:29.720 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T01:30:29.720 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T01:30:29.720 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T01:30:29.720 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T01:30:29.720 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T01:30:29.720 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T01:30:29.720 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T01:30:29.720 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T01:30:29.720 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T01:30:29.720 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T01:30:29.720 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:29.721 INFO:teuthology.orchestra.run.smithi175.stdout:Install 34 Packages 2024-09-15T01:30:29.722 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:29.722 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 27 M 2024-09-15T01:30:29.722 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 96 M 2024-09-15T01:30:29.722 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:30:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:30:30.242 INFO:teuthology.orchestra.run.smithi175.stdout:(1/34): ceph-17.2.7-1643.gd75f2382.el9.x86_64.r 23 kB/s | 6.5 kB 00:00 2024-09-15T01:30:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-15T01:30:30.285 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-15T01:30:30.318 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-15T01:30:30.327 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-15T01:30:30.356 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-15T01:30:30.356 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:30.356 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T01:30:30.356 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T01:30:30.356 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T01:30:30.356 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-15T01:30:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-15T01:30:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-15T01:30:30.581 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-15T01:30:30.611 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-15T01:30:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-15T01:30:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-15T01:30:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-15T01:30:30.737 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-15T01:30:30.761 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-15T01:30:30.761 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:30.761 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T01:30:30.762 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T01:30:30.762 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T01:30:30.762 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-15T01:30:30.800 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-15T01:30:30.850 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-15T01:30:30.890 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T01:30:30.962 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-15T01:30:30.975 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-15T01:30:30.986 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-15T01:30:30.986 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:30.986 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T01:30:30.986 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T01:30:30.986 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T01:30:30.986 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-15T01:30:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 1/34 2024-09-15T01:30:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2/34 2024-09-15T01:30:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 3/34 2024-09-15T01:30:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 4/34 2024-09-15T01:30:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 5/34 2024-09-15T01:30:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 6/34 2024-09-15T01:30:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-15T01:30:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-15T01:30:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-15T01:30:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-15T01:30:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-15T01:30:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-15T01:30:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-15T01:30:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-15T01:30:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T01:30:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-15T01:30:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-15T01:30:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-15T01:30:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-15T01:30:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-15T01:30:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-15T01:30:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-15T01:30:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-15T01:30:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-15T01:30:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-15T01:30:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-15T01:30:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-15T01:30:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-15T01:30:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-15T01:30:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-15T01:30:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-15T01:30:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-15T01:30:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-15T01:30:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-15T01:30:31.127 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-15T01:30:31.209 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-15T01:30:31.260 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-15T01:30:31.297 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-15T01:30:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-15T01:30:31.417 INFO:teuthology.orchestra.run.smithi175.stdout:(2/34): ceph-mgr-17.2.7-1643.gd75f2382.el9.x86_ 988 kB/s | 1.4 MB 00:01 2024-09-15T01:30:31.436 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-15T01:30:31.632 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-15T01:30:31.685 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-15T01:30:31.725 INFO:teuthology.orchestra.run.smithi175.stdout:(3/34): ceph-mds-17.2.7-1643.gd75f2382.el9.x86_ 1.2 MB/s | 2.1 MB 00:01 2024-09-15T01:30:31.774 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-15T01:30:31.834 INFO:teuthology.orchestra.run.smithi175.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1643.gd75f 2.2 MB/s | 240 kB 00:00 2024-09-15T01:30:31.872 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-15T01:30:31.909 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-15T01:30:31.951 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-15T01:30:31.952 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:31.952 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:30:31.952 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:31.952 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:31.952 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:31.952 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:31.952 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:31.952 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:31.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T01:30:31.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T01:30:31.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T01:30:31.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T01:30:31.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:31.954 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:30:31.968 INFO:teuthology.orchestra.run.smithi175.stdout:(5/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-09-15T01:30:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-15T01:30:32.052 INFO:teuthology.orchestra.run.smithi175.stdout:(6/34): python3-cryptography-36.0.1-4.el9.x86_6 15 MB/s | 1.2 MB 00:00 2024-09-15T01:30:32.094 INFO:teuthology.orchestra.run.smithi175.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-15T01:30:32.135 INFO:teuthology.orchestra.run.smithi175.stdout:(8/34): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-09-15T01:30:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 29/34 2024-09-15T01:30:32.194 INFO:teuthology.orchestra.run.smithi175.stdout:(9/34): ceph-mon-17.2.7-1643.gd75f2382.el9.x86_ 2.0 MB/s | 3.9 MB 00:01 2024-09-15T01:30:32.210 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-15T01:30:32.219 INFO:teuthology.orchestra.run.smithi175.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 1.5 MB/s | 125 kB 00:00 2024-09-15T01:30:32.237 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-15T01:30:32.237 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:32.237 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T01:30:32.237 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T01:30:32.238 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T01:30:32.238 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:32.320 INFO:teuthology.orchestra.run.smithi175.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 1.7 MB/s | 215 kB 00:00 2024-09-15T01:30:32.322 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-base 2024-09-15T01:30:32.411 INFO:teuthology.orchestra.run.smithi175.stdout:(12/34): python3-mako-1.1.4-6.el9.noarch.rpm 898 kB/s | 172 kB 00:00 2024-09-15T01:30:32.437 INFO:teuthology.orchestra.run.smithi175.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 297 kB/s | 35 kB 00:00 2024-09-15T01:30:32.462 INFO:teuthology.orchestra.run.smithi175.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-15T01:30:32.512 INFO:teuthology.orchestra.run.smithi175.stdout:(15/34): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-09-15T01:30:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-09-15T01:30:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:(17/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 649 kB/s | 43 kB 00:00 2024-09-15T01:30:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:(18/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.6 MB/s | 172 kB 00:00 2024-09-15T01:30:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:(19/34): python3-cherrypy-18.6.1-2.el9.noarch.r 6.2 MB/s | 358 kB 00:00 2024-09-15T01:30:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:(20/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 212 kB/s | 11 kB 00:00 2024-09-15T01:30:32.692 INFO:teuthology.orchestra.run.smithi175.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 331 kB/s | 18 kB 00:00 2024-09-15T01:30:32.717 INFO:teuthology.orchestra.run.smithi175.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 433 kB/s | 23 kB 00:00 2024-09-15T01:30:32.742 INFO:teuthology.orchestra.run.smithi175.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-15T01:30:32.768 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-15T01:30:32.768 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 1/34 2024-09-15T01:30:32.768 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2/34 2024-09-15T01:30:32.768 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 3/34 2024-09-15T01:30:32.768 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 4/34 2024-09-15T01:30:32.768 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 5/34 2024-09-15T01:30:32.768 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 6/34 2024-09-15T01:30:32.768 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-15T01:30:32.769 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-15T01:30:32.770 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-15T01:30:32.770 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-15T01:30:32.770 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-15T01:30:32.770 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-15T01:30:32.770 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-15T01:30:32.770 INFO:teuthology.orchestra.run.smithi175.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-15T01:30:32.771 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-15T01:30:32.771 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-15T01:30:32.771 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-15T01:30:32.772 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-15T01:30:32.772 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-15T01:30:32.772 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-15T01:30:32.772 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-15T01:30:32.772 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-15T01:30:32.793 INFO:teuthology.orchestra.run.smithi175.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-15T01:30:32.827 INFO:teuthology.orchestra.run.smithi175.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-15T01:30:32.844 INFO:teuthology.orchestra.run.smithi175.stdout:(27/34): python3-portend-3.1.0-2.el9.noarch.rpm 1.0 MB/s | 16 kB 00:00 2024-09-15T01:30:32.877 INFO:teuthology.orchestra.run.smithi175.stdout:(28/34): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-15T01:30:32.891 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:50 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:32.902 INFO:teuthology.orchestra.run.smithi175.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-15T01:30:32.928 INFO:teuthology.orchestra.run.smithi175.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-09-15T01:30:32.962 INFO:teuthology.orchestra.run.smithi175.stdout:(31/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-09-15T01:30:32.981 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:30:32.987 INFO:teuthology.orchestra.run.smithi175.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 802 kB/s | 20 kB 00:00 2024-09-15T01:30:33.012 INFO:teuthology.orchestra.run.smithi175.stdout:(33/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.0 MB/s | 427 kB 00:00 2024-09-15T01:30:33.023 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:30:33.024 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-15T01:30:33.024 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:30:33.098 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install cephadm 2024-09-15T01:30:33.254 INFO:teuthology.orchestra.run.smithi175.stdout:(34/34): ceph-osd-17.2.7-1643.gd75f2382.el9.x86 8.4 MB/s | 15 MB 00:01 2024-09-15T01:30:33.257 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:33.257 INFO:teuthology.orchestra.run.smithi175.stdout:Total 7.7 MB/s | 27 MB 00:03 2024-09-15T01:30:33.384 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:30:33.421 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:30:33.422 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:30:33.653 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-15T01:30:33.661 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:51 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:33.682 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-15T01:30:33.682 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:33.682 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T01:30:33.682 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T01:30:33.682 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T01:30:33.682 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-15T01:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T01:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T01:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T01:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T01:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T01:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:30:33.793 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:30:33.794 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-15T01:30:33.794 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:33.794 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:30:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 83 k 2024-09-15T01:30:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:33.794 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:30:33.794 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:33.794 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-15T01:30:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:33.794 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 83 k 2024-09-15T01:30:33.794 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 375 k 2024-09-15T01:30:33.795 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:30:33.808 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:30:33.809 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:30:33.981 INFO:teuthology.orchestra.run.smithi115.stdout:cephadm-17.2.7-1643.gd75f2382.el9.noarch.rpm 445 kB/s | 83 kB 00:00 2024-09-15T01:30:33.981 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:33.982 INFO:teuthology.orchestra.run.smithi115.stdout:Total 441 kB/s | 83 kB 00:00 2024-09-15T01:30:33.982 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:30:33.987 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:30:33.987 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:30:33.989 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:30:33.989 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:30:34.054 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:30:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-15T01:30:34.092 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-base 2024-09-15T01:30:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-15T01:30:34.112 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:34.112 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T01:30:34.112 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T01:30:34.112 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T01:30:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:34.329 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-15T01:30:34.354 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-15T01:30:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T01:30:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T01:30:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T01:30:34.355 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:34.421 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-15T01:30:34.422 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:34.526 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:34.666 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:52 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:Package ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:30:34.796 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:30:34.797 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-09-15T01:30:34.798 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:30:34.857 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install cephadm 2024-09-15T01:30:34.859 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:35.092 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:30:35.114 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:35.114 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:35.114 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:30:35.114 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:35.114 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:35.114 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:30:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-15T01:30:35.197 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-15T01:30:35.238 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-15T01:30:35.330 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-15T01:30:35.353 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T01:30:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-15T01:30:35.422 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:53 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-15T01:30:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-15T01:30:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-15T01:30:35.554 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:30:35.554 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:35.555 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-09-15T01:30:35.555 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:35.555 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:30:35.555 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 83 k 2024-09-15T01:30:35.555 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:35.555 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:30:35.555 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:35.555 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-09-15T01:30:35.555 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:35.555 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 83 k 2024-09-15T01:30:35.555 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 375 k 2024-09-15T01:30:35.555 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:30:35.557 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-15T01:30:35.599 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-15T01:30:35.639 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-15T01:30:35.674 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-15T01:30:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-15T01:30:35.751 INFO:teuthology.orchestra.run.smithi152.stdout:cephadm-17.2.7-1643.gd75f2382.el9.noarch.rpm 423 kB/s | 83 kB 00:00 2024-09-15T01:30:35.752 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:35.752 INFO:teuthology.orchestra.run.smithi152.stdout:Total 419 kB/s | 83 kB 00:00 2024-09-15T01:30:35.752 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:30:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-15T01:30:35.757 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:30:35.758 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:30:35.759 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:30:35.760 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:30:35.798 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T01:30:35.835 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:30:35.869 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-15T01:30:35.930 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:53 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:36.010 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-15T01:30:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 1/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 3/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 4/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 5/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 6/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-15T01:30:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-15T01:30:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-15T01:30:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T01:30:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-15T01:30:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-15T01:30:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-15T01:30:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-15T01:30:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-15T01:30:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-15T01:30:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-15T01:30:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-15T01:30:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-15T01:30:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-15T01:30:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-15T01:30:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-15T01:30:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-15T01:30:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-15T01:30:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-15T01:30:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-15T01:30:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-15T01:30:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-15T01:30:36.064 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:30:36.065 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:36.065 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-15T01:30:36.065 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:36.065 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:30:36.065 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 137 k 2024-09-15T01:30:36.065 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:36.065 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:30:36.065 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:36.065 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-15T01:30:36.066 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:36.066 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 137 k 2024-09-15T01:30:36.066 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 379 k 2024-09-15T01:30:36.066 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:30:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-15T01:30:36.150 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-15T01:30:36.189 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-15T01:30:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-15T01:30:36.269 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-immutable-object-cache-17.2.7-1643.gd75f23 676 kB/s | 137 kB 00:00 2024-09-15T01:30:36.270 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:36.270 INFO:teuthology.orchestra.run.smithi115.stdout:Total 671 kB/s | 137 kB 00:00 2024-09-15T01:30:36.270 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:30:36.277 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:30:36.277 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:30:36.302 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:36.316 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-15T01:30:36.333 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:30:36.334 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:30:36.387 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:36.476 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:30:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-15T01:30:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:36.561 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:36.562 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T01:30:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-15T01:30:36.645 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-15T01:30:36.709 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:36.741 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-15T01:30:36.791 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-15T01:30:36.914 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-15T01:30:36.943 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:36.943 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:36.944 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:30:36.944 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:36.944 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:36.944 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:30:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-15T01:30:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:37.050 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:30:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T01:30:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T01:30:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T01:30:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T01:30:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T01:30:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T01:30:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T01:30:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T01:30:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T01:30:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T01:30:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T01:30:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T01:30:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T01:30:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T01:30:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T01:30:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T01:30:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:30:37.069 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 29/34 2024-09-15T01:30:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-15T01:30:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-15T01:30:37.136 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:37.136 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T01:30:37.136 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T01:30:37.136 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T01:30:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:37.145 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T01:30:37.410 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-base 2024-09-15T01:30:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:37.514 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:30:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:37.514 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:30:37.706 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr 2024-09-15T01:30:37.719 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:55 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:37.856 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:30:37.856 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:37.857 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2024-09-15T01:30:37.857 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:37.857 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:30:37.857 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 137 k 2024-09-15T01:30:37.857 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:37.857 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:30:37.857 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:37.857 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-09-15T01:30:37.857 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:37.858 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 137 k 2024-09-15T01:30:37.858 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 379 k 2024-09-15T01:30:37.858 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:30:37.997 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:53 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:30:38.040 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-immutable-object-cache-17.2.7-1643.gd75f23 757 kB/s | 137 kB 00:00 2024-09-15T01:30:38.040 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:38.041 INFO:teuthology.orchestra.run.smithi152.stdout:Total 750 kB/s | 137 kB 00:00 2024-09-15T01:30:38.041 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:30:38.048 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:30:38.048 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:30:38.090 INFO:teuthology.orchestra.run.smithi042.stdout:Package ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:30:38.105 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:30:38.105 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:30:38.133 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:30:38.135 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-15T01:30:38.135 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:30:38.211 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install cephadm 2024-09-15T01:30:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:30:38.283 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:56 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:38.343 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:38.368 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:38.368 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T01:30:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:38.373 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:30:38.414 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:30:38.415 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-15T01:30:38.415 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:30:38.443 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-15T01:30:38.471 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-15T01:30:38.471 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:38.471 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T01:30:38.471 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T01:30:38.471 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T01:30:38.471 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:38.487 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T01:30:38.800 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:54 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:30:38.852 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-15T01:30:38.876 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-15T01:30:38.876 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:38.876 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T01:30:38.876 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T01:30:38.876 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T01:30:38.876 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:38.938 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:30:38.939 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-15T01:30:38.939 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:38.939 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:30:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 83 k 2024-09-15T01:30:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:38.940 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:30:38.940 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:38.940 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-15T01:30:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:38.940 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 83 k 2024-09-15T01:30:38.940 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 375 k 2024-09-15T01:30:38.940 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:30:39.052 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:57 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:39.087 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-15T01:30:39.111 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-15T01:30:39.111 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:39.111 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T01:30:39.111 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T01:30:39.111 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T01:30:39.111 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:39.137 INFO:teuthology.orchestra.run.smithi042.stdout:cephadm-17.2.7-1643.gd75f2382.el9.noarch.rpm 420 kB/s | 83 kB 00:00 2024-09-15T01:30:39.138 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:39.138 INFO:teuthology.orchestra.run.smithi042.stdout:Total 417 kB/s | 83 kB 00:00 2024-09-15T01:30:39.138 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:30:39.143 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:30:39.144 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:30:39.145 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:30:39.146 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:30:39.175 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-15T01:30:39.185 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:30:39.186 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:39.186 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-15T01:30:39.186 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:39.186 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:30:39.186 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 3.4 M 2024-09-15T01:30:39.186 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-15T01:30:39.187 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 24 k 2024-09-15T01:30:39.187 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 15 k 2024-09-15T01:30:39.187 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-15T01:30:39.187 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T01:30:39.187 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T01:30:39.187 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-09-15T01:30:39.187 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-15T01:30:39.187 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:39.187 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:30:39.187 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:39.187 INFO:teuthology.orchestra.run.smithi115.stdout:Install 7 Packages 2024-09-15T01:30:39.187 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:39.188 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 3.7 M 2024-09-15T01:30:39.188 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 80 M 2024-09-15T01:30:39.188 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:30:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:30:39.283 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:39.283 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:39.283 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:30:39.283 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:39.283 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:39.283 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:30:39.431 INFO:teuthology.orchestra.run.smithi115.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1643.gd75f 69 kB/s | 15 kB 00:00 2024-09-15T01:30:39.457 INFO:teuthology.orchestra.run.smithi115.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1643.gd75 103 kB/s | 24 kB 00:00 2024-09-15T01:30:39.490 INFO:teuthology.orchestra.run.smithi115.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 155 kB/s | 9.0 kB 00:00 2024-09-15T01:30:39.494 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr 2024-09-15T01:30:39.515 INFO:teuthology.orchestra.run.smithi115.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-15T01:30:39.541 INFO:teuthology.orchestra.run.smithi115.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 489 kB/s | 41 kB 00:00 2024-09-15T01:30:39.566 INFO:teuthology.orchestra.run.smithi115.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-15T01:30:39.681 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:39.691 INFO:teuthology.orchestra.run.smithi115.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1643.gd75f2382 7.1 MB/s | 3.4 MB 00:00 2024-09-15T01:30:39.695 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:39.695 INFO:teuthology.orchestra.run.smithi115.stdout:Total 7.2 MB/s | 3.7 MB 00:00 2024-09-15T01:30:39.695 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:30:39.704 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:30:39.704 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:30:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:39.788 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:30:39.788 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:30:40.069 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:58 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:40.090 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:30:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:40.151 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-15T01:30:40.160 INFO:teuthology.orchestra.run.smithi152.stdout:Package ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:30:40.193 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-15T01:30:40.202 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:30:40.203 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-09-15T01:30:40.204 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:30:40.240 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-15T01:30:40.265 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T01:30:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-15T01:30:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 5/7 2024-09-15T01:30:40.396 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:40.396 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:40.396 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:30:40.396 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:40.396 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:40.396 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:30:40.641 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T01:30:40.773 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-15T01:30:40.773 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 1/34 2024-09-15T01:30:40.773 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 3/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 4/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 5/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 6/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T01:30:40.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-15T01:30:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-15T01:30:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-15T01:30:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-15T01:30:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-15T01:30:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-15T01:30:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-15T01:30:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-15T01:30:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-15T01:30:40.775 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-15T01:30:40.776 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-15T01:30:40.776 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-15T01:30:40.776 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-15T01:30:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-15T01:30:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-15T01:30:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-15T01:30:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-15T01:30:40.777 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-15T01:30:40.849 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:58 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:40.984 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:30:40.984 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-09-15T01:30:40.985 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:40.985 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:30:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 3.4 M 2024-09-15T01:30:40.985 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-09-15T01:30:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 24 k 2024-09-15T01:30:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 15 k 2024-09-15T01:30:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-15T01:30:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T01:30:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T01:30:40.985 INFO:teuthology.orchestra.run.smithi152.stdout:Installing weak dependencies: 2024-09-15T01:30:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-15T01:30:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:40.986 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:30:40.986 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:40.986 INFO:teuthology.orchestra.run.smithi152.stdout:Install 7 Packages 2024-09-15T01:30:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:40.986 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 3.7 M 2024-09-15T01:30:40.986 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 80 M 2024-09-15T01:30:40.986 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1643.gd75 161 kB/s | 24 kB 00:00 2024-09-15T01:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1643.gd75f 83 kB/s | 15 kB 00:00 2024-09-15T01:30:41.223 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:57 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 154 kB/s | 9.0 kB 00:00 2024-09-15T01:30:41.243 INFO:teuthology.orchestra.run.smithi152.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 2.0 MB/s | 31 kB 00:00 2024-09-15T01:30:41.260 INFO:teuthology.orchestra.run.smithi152.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 609 kB/s | 41 kB 00:00 2024-09-15T01:30:41.285 INFO:teuthology.orchestra.run.smithi152.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-09-15T01:30:41.299 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 6/7 2024-09-15T01:30:41.339 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-15T01:30:41.359 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:30:41.360 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:41.360 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-09-15T01:30:41.360 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:41.360 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:30:41.360 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 137 k 2024-09-15T01:30:41.360 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:41.360 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:30:41.360 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:41.361 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-15T01:30:41.361 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:41.361 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 137 k 2024-09-15T01:30:41.361 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 379 k 2024-09-15T01:30:41.361 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:30:41.436 INFO:teuthology.orchestra.run.smithi152.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1643.gd75f2382 8.0 MB/s | 3.4 MB 00:00 2024-09-15T01:30:41.439 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:41.439 INFO:teuthology.orchestra.run.smithi152.stdout:Total 8.0 MB/s | 3.7 MB 00:00 2024-09-15T01:30:41.440 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:30:41.448 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:30:41.448 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:30:41.531 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:30:41.531 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:30:41.553 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-immutable-object-cache-17.2.7-1643.gd75f23 716 kB/s | 137 kB 00:00 2024-09-15T01:30:41.553 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:41.554 INFO:teuthology.orchestra.run.smithi042.stdout:Total 710 kB/s | 137 kB 00:00 2024-09-15T01:30:41.554 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:30:41.562 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:30:41.562 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:30:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:30:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:30:41.774 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:30:41.832 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:30:41.846 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:41.853 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-15T01:30:41.853 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T01:30:41.854 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T01:30:41.855 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T01:30:41.855 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T01:30:41.855 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T01:30:41.855 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T01:30:41.855 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T01:30:41.855 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T01:30:41.855 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T01:30:41.855 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T01:30:41.855 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T01:30:41.855 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T01:30:41.856 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T01:30:41.856 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T01:30:41.856 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T01:30:41.856 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T01:30:41.856 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T01:30:41.856 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T01:30:41.856 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T01:30:41.856 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T01:30:41.856 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T01:30:41.856 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T01:30:41.856 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T01:30:41.856 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T01:30:41.857 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:41.857 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:30:41.872 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:41.872 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:41.873 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T01:30:41.873 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:41.903 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-15T01:30:41.938 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-15T01:30:41.984 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-15T01:30:42.016 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-15T01:30:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 5/7 2024-09-15T01:30:42.191 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-base 2024-09-15T01:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-15T01:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 1/7 2024-09-15T01:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 2/7 2024-09-15T01:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 3/7 2024-09-15T01:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-15T01:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-15T01:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-15T01:30:42.620 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-15T01:30:42.620 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:42.621 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:30:42.621 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:42.621 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:42.621 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:42.621 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-15T01:30:42.621 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-15T01:30:42.621 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T01:30:42.621 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T01:30:42.621 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:42.621 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:30:42.765 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:54 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:30:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:42.796 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:30:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:42.797 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:30:42.831 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T01:30:42.855 INFO:teuthology.orchestra.run.smithi175.stdout:Package ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:30:42.897 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:30:42.898 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-15T01:30:42.898 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:30:42.964 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install cephadm 2024-09-15T01:30:43.042 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 6/7 2024-09-15T01:30:43.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr 2024-09-15T01:30:43.081 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-15T01:30:43.392 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:01 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:55 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:30:43.525 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:30:43.527 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-09-15T01:30:43.527 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-15T01:30:43.527 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-09-15T01:30:43.527 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:30:43.527 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 7.4 M 2024-09-15T01:30:43.527 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-15T01:30:43.527 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T01:30:43.527 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T01:30:43.527 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T01:30:43.527 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T01:30:43.527 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T01:30:43.527 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T01:30:43.528 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T01:30:43.528 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T01:30:43.528 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T01:30:43.528 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T01:30:43.528 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T01:30:43.528 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:43.528 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:30:43.528 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-09-15T01:30:43.528 INFO:teuthology.orchestra.run.smithi115.stdout:Install 12 Packages 2024-09-15T01:30:43.528 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:43.529 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 38 M 2024-09-15T01:30:43.529 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 212 M 2024-09-15T01:30:43.529 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:30:43.649 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:59 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:30:43.655 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:30:43.655 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:43.655 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-15T01:30:43.655 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:43.655 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:30:43.655 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 83 k 2024-09-15T01:30:43.656 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:43.656 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:30:43.656 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:43.656 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-15T01:30:43.656 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:43.656 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 83 k 2024-09-15T01:30:43.656 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 375 k 2024-09-15T01:30:43.657 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:30:43.743 INFO:teuthology.orchestra.run.smithi042.stdout:Package ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:30:43.789 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:30:43.790 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-15T01:30:43.790 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:30:43.809 INFO:teuthology.orchestra.run.smithi115.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-15T01:30:43.826 INFO:teuthology.orchestra.run.smithi175.stdout:cephadm-17.2.7-1643.gd75f2382.el9.noarch.rpm 487 kB/s | 83 kB 00:00 2024-09-15T01:30:43.826 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:43.827 INFO:teuthology.orchestra.run.smithi175.stdout:Total 484 kB/s | 83 kB 00:00 2024-09-15T01:30:43.827 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:30:43.832 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:30:43.832 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:30:43.834 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:30:43.834 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:30:43.844 INFO:teuthology.orchestra.run.smithi115.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.4 MB/s | 801 kB 00:00 2024-09-15T01:30:43.873 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T01:30:43.906 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:30:43.909 INFO:teuthology.orchestra.run.smithi115.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-09-15T01:30:43.951 INFO:teuthology.orchestra.run.smithi115.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 438 kB/s | 18 kB 00:00 2024-09-15T01:30:44.010 INFO:teuthology.orchestra.run.smithi115.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 659 kB/s | 38 kB 00:00 2024-09-15T01:30:44.085 INFO:teuthology.orchestra.run.smithi115.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-09-15T01:30:44.114 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-15T01:30:44.115 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 1/7 2024-09-15T01:30:44.115 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 2/7 2024-09-15T01:30:44.115 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 3/7 2024-09-15T01:30:44.115 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-15T01:30:44.115 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-15T01:30:44.115 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-15T01:30:44.194 INFO:teuthology.orchestra.run.smithi115.stdout:(7/12): ceph-mgr-diskprediction-local-17.2.7-16 14 MB/s | 7.4 MB 00:00 2024-09-15T01:30:44.219 INFO:teuthology.orchestra.run.smithi115.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-15T01:30:44.261 INFO:teuthology.orchestra.run.smithi115.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 5.9 MB/s | 250 kB 00:00 2024-09-15T01:30:44.353 INFO:teuthology.orchestra.run.smithi115.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 14 MB/s | 4.9 MB 00:00 2024-09-15T01:30:44.388 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-15T01:30:44.388 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:44.388 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:30:44.388 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:44.388 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:44.388 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:44.388 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-15T01:30:44.388 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-15T01:30:44.389 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T01:30:44.389 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T01:30:44.389 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:44.389 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:30:44.400 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:44.457 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:00 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:30:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:44.595 INFO:teuthology.orchestra.run.smithi115.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 49 MB/s | 16 MB 00:00 2024-09-15T01:30:44.596 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:30:44.597 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T01:30:44.597 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-15T01:30:44.598 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:44.598 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:30:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 3.4 M 2024-09-15T01:30:44.598 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-15T01:30:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 24 k 2024-09-15T01:30:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 15 k 2024-09-15T01:30:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-15T01:30:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T01:30:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T01:30:44.598 INFO:teuthology.orchestra.run.smithi042.stdout:Installing weak dependencies: 2024-09-15T01:30:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-15T01:30:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:44.599 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:30:44.599 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:30:44.599 INFO:teuthology.orchestra.run.smithi042.stdout:Install 7 Packages 2024-09-15T01:30:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:44.599 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 3.7 M 2024-09-15T01:30:44.599 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 80 M 2024-09-15T01:30:44.599 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:30:44.670 INFO:teuthology.orchestra.run.smithi115.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-15T01:30:44.671 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:44.671 INFO:teuthology.orchestra.run.smithi115.stdout:Total 33 MB/s | 38 MB 00:01 2024-09-15T01:30:44.782 INFO:teuthology.orchestra.run.smithi042.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1643.gd75f 97 kB/s | 15 kB 00:00 2024-09-15T01:30:44.807 INFO:teuthology.orchestra.run.smithi042.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1643.gd75 139 kB/s | 24 kB 00:00 2024-09-15T01:30:44.847 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:44.849 INFO:teuthology.orchestra.run.smithi042.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-15T01:30:44.874 INFO:teuthology.orchestra.run.smithi042.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-15T01:30:44.899 INFO:teuthology.orchestra.run.smithi042.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-15T01:30:44.924 INFO:teuthology.orchestra.run.smithi042.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-15T01:30:45.025 INFO:teuthology.orchestra.run.smithi042.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1643.gd75f2382 8.5 MB/s | 3.4 MB 00:00 2024-09-15T01:30:45.027 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:30:45.028 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:45.029 INFO:teuthology.orchestra.run.smithi042.stdout:Total 8.5 MB/s | 3.7 MB 00:00 2024-09-15T01:30:45.029 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:30:45.038 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:30:45.038 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:30:45.052 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:30:45.052 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:30:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:45.090 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:30:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:45.090 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:30:45.128 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:30:45.128 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:30:45.184 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:03 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:45.281 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T01:30:45.328 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:30:45.329 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================= 2024-09-15T01:30:45.329 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-09-15T01:30:45.329 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================= 2024-09-15T01:30:45.329 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:30:45.329 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 7.4 M 2024-09-15T01:30:45.329 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-09-15T01:30:45.330 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T01:30:45.330 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T01:30:45.330 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T01:30:45.330 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T01:30:45.330 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T01:30:45.330 INFO:teuthology.orchestra.run.smithi152.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T01:30:45.330 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T01:30:45.330 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T01:30:45.330 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T01:30:45.331 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T01:30:45.331 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T01:30:45.331 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:45.331 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:30:45.331 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================= 2024-09-15T01:30:45.331 INFO:teuthology.orchestra.run.smithi152.stdout:Install 12 Packages 2024-09-15T01:30:45.332 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:45.332 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 38 M 2024-09-15T01:30:45.332 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 212 M 2024-09-15T01:30:45.332 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:30:45.442 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:30:45.501 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-15T01:30:45.536 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-15T01:30:45.545 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:30:45.545 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:30:45.584 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-15T01:30:45.614 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-15T01:30:45.643 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 5/7 2024-09-15T01:30:45.855 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:57 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:30:45.990 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:30:45.991 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:45.991 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2024-09-15T01:30:45.991 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:45.992 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:30:45.992 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 137 k 2024-09-15T01:30:45.992 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:45.992 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:30:45.992 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:45.992 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-15T01:30:45.992 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:45.992 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 137 k 2024-09-15T01:30:45.992 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 379 k 2024-09-15T01:30:45.992 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:30:46.135 INFO:teuthology.orchestra.run.smithi152.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 17 MB/s | 7.4 MB 00:00 2024-09-15T01:30:46.190 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-immutable-object-cache-17.2.7-1643.gd75f23 697 kB/s | 137 kB 00:00 2024-09-15T01:30:46.190 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:46.191 INFO:teuthology.orchestra.run.smithi175.stdout:Total 690 kB/s | 137 kB 00:00 2024-09-15T01:30:46.191 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:30:46.198 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:30:46.198 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:30:46.235 INFO:teuthology.orchestra.run.smithi152.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 352 kB/s | 191 kB 00:00 2024-09-15T01:30:46.254 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:30:46.254 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:30:46.260 INFO:teuthology.orchestra.run.smithi152.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 262 kB/s | 33 kB 00:00 2024-09-15T01:30:46.295 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:30:46.343 INFO:teuthology.orchestra.run.smithi152.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-09-15T01:30:46.363 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T01:30:46.401 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:30:46.410 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T01:30:46.418 INFO:teuthology.orchestra.run.smithi152.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 514 kB/s | 38 kB 00:00 2024-09-15T01:30:46.499 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 988 kB/s | 801 kB 00:00 2024-09-15T01:30:46.507 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T01:30:46.524 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:46.524 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:46.524 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T01:30:46.524 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:46.561 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T01:30:46.669 INFO:teuthology.orchestra.run.smithi152.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-09-15T01:30:46.695 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 6/7 2024-09-15T01:30:46.737 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-15T01:30:46.745 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T01:30:46.777 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T01:30:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T01:30:46.994 INFO:teuthology.orchestra.run.smithi152.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.9 MB/s | 3.0 MB 00:00 2024-09-15T01:30:47.070 INFO:teuthology.orchestra.run.smithi152.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.3 MB/s | 250 kB 00:00 2024-09-15T01:30:47.399 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:47.399 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:47.400 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:30:47.400 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:47.400 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:47.400 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:30:47.493 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T01:30:47.614 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T01:30:47.617 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr 2024-09-15T01:30:47.753 INFO:teuthology.orchestra.run.smithi152.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.7 MB/s | 5.1 MB 00:01 2024-09-15T01:30:47.823 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-15T01:30:47.823 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 1/7 2024-09-15T01:30:47.823 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 2/7 2024-09-15T01:30:47.823 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 3/7 2024-09-15T01:30:47.823 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-15T01:30:47.823 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-15T01:30:47.823 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-15T01:30:48.070 INFO:teuthology.orchestra.run.smithi152.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.0 MB/s | 4.9 MB 00:01 2024-09-15T01:30:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-15T01:30:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:48.104 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:30:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-15T01:30:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-15T01:30:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T01:30:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T01:30:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:48.105 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:30:48.195 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:00 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:30:48.289 INFO:teuthology.orchestra.run.smithi175.stdout:Package ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:30:48.332 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:30:48.333 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-15T01:30:48.333 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:30:48.363 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T01:30:48.393 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T01:30:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T01:30:48.946 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:00 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:30:48.953 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:04 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:30:49.079 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:30:49.079 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-15T01:30:49.080 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:49.080 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:30:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 3.4 M 2024-09-15T01:30:49.080 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-15T01:30:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 24 k 2024-09-15T01:30:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 15 k 2024-09-15T01:30:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-15T01:30:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T01:30:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T01:30:49.081 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-09-15T01:30:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-15T01:30:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:49.081 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:30:49.081 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:30:49.081 INFO:teuthology.orchestra.run.smithi175.stdout:Install 7 Packages 2024-09-15T01:30:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:49.082 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 3.7 M 2024-09-15T01:30:49.082 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 80 M 2024-09-15T01:30:49.082 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:30:49.094 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:30:49.095 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-09-15T01:30:49.095 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-15T01:30:49.095 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-09-15T01:30:49.096 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:30:49.096 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 7.4 M 2024-09-15T01:30:49.096 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-15T01:30:49.096 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T01:30:49.096 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T01:30:49.096 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T01:30:49.096 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T01:30:49.096 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T01:30:49.096 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T01:30:49.096 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T01:30:49.096 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T01:30:49.096 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T01:30:49.097 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T01:30:49.097 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T01:30:49.097 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:49.097 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:30:49.097 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-09-15T01:30:49.097 INFO:teuthology.orchestra.run.smithi042.stdout:Install 12 Packages 2024-09-15T01:30:49.097 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 38 M 2024-09-15T01:30:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 212 M 2024-09-15T01:30:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:30:49.393 INFO:teuthology.orchestra.run.smithi175.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1643.gd75 160 kB/s | 24 kB 00:00 2024-09-15T01:30:49.418 INFO:teuthology.orchestra.run.smithi175.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1643.gd75f 82 kB/s | 15 kB 00:00 2024-09-15T01:30:49.462 INFO:teuthology.orchestra.run.smithi175.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 131 kB/s | 9.0 kB 00:00 2024-09-15T01:30:49.487 INFO:teuthology.orchestra.run.smithi175.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-15T01:30:49.513 INFO:teuthology.orchestra.run.smithi175.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 434 kB/s | 41 kB 00:00 2024-09-15T01:30:49.538 INFO:teuthology.orchestra.run.smithi175.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-15T01:30:49.691 INFO:teuthology.orchestra.run.smithi042.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 17 MB/s | 7.4 MB 00:00 2024-09-15T01:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.1 MB/s | 16 MB 00:02 2024-09-15T01:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout:Total 8.7 MB/s | 38 MB 00:04 2024-09-15T01:30:49.741 INFO:teuthology.orchestra.run.smithi042.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 383 kB/s | 191 kB 00:00 2024-09-15T01:30:49.775 INFO:teuthology.orchestra.run.smithi042.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 392 kB/s | 33 kB 00:00 2024-09-15T01:30:49.817 INFO:teuthology.orchestra.run.smithi042.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 436 kB/s | 18 kB 00:00 2024-09-15T01:30:49.846 INFO:teuthology.orchestra.run.smithi175.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1643.gd75f2382 5.5 MB/s | 3.4 MB 00:00 2024-09-15T01:30:49.850 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:49.850 INFO:teuthology.orchestra.run.smithi175.stdout:Total 4.7 MB/s | 3.7 MB 00:00 2024-09-15T01:30:49.850 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:30:49.858 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:30:49.858 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:30:49.859 INFO:teuthology.orchestra.run.smithi042.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 920 kB/s | 38 kB 00:00 2024-09-15T01:30:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T01:30:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-15T01:30:49.926 INFO:teuthology.orchestra.run.smithi042.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-15T01:30:49.940 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:30:49.940 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:30:50.001 INFO:teuthology.orchestra.run.smithi042.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-09-15T01:30:50.085 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:30:50.110 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:30:50.110 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:30:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-09-15T01:30:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2024-09-15T01:30:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 4.9 MB/s | 250 kB 00:00 2024-09-15T01:30:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:30:50.260 INFO:teuthology.orchestra.run.smithi042.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 12 MB/s | 4.9 MB 00:00 2024-09-15T01:30:50.364 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-15T01:30:50.398 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-15T01:30:50.452 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-15T01:30:50.510 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-15T01:30:50.539 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 5/7 2024-09-15T01:30:50.611 INFO:teuthology.orchestra.run.smithi042.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-09-15T01:30:50.611 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:50.612 INFO:teuthology.orchestra.run.smithi042.stdout:Total 25 MB/s | 38 MB 00:01 2024-09-15T01:30:50.617 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:30:50.617 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:30:50.977 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:30:51.004 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:30:51.004 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:30:51.379 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:30:51.456 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T01:30:51.498 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T01:30:51.525 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 6/7 2024-09-15T01:30:51.534 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:30:51.534 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:30:51.559 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-15T01:30:51.593 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T01:30:51.636 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T01:30:51.824 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T01:30:51.859 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T01:30:51.938 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T01:30:52.320 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:30:52.400 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T01:30:52.449 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T01:30:52.502 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-15T01:30:52.502 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 1/12 2024-09-15T01:30:52.502 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T01:30:52.502 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T01:30:52.502 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T01:30:52.502 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T01:30:52.502 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T01:30:52.502 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T01:30:52.502 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T01:30:52.502 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T01:30:52.503 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T01:30:52.503 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T01:30:52.555 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T01:30:52.587 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T01:30:52.594 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T01:30:52.638 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-15T01:30:52.639 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 1/7 2024-09-15T01:30:52.639 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 2/7 2024-09-15T01:30:52.639 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 3/7 2024-09-15T01:30:52.639 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-15T01:30:52.639 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-15T01:30:52.639 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-15T01:30:52.707 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T01:30:52.788 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T01:30:52.826 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T01:30:52.912 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T01:30:52.915 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-15T01:30:52.915 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:52.915 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:30:52.915 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:52.915 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:52.915 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:52.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-15T01:30:52.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-15T01:30:52.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T01:30:52.915 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T01:30:52.915 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:52.915 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:30:52.920 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T01:30:52.921 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:52.921 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:30:52.921 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:52.921 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T01:30:52.921 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T01:30:52.921 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T01:30:52.921 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T01:30:52.921 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T01:30:52.921 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T01:30:52.921 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T01:30:52.921 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T01:30:52.921 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T01:30:52.922 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T01:30:52.922 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T01:30:52.922 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:52.922 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:30:53.112 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T01:30:53.361 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-rook 2024-09-15T01:30:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T01:30:53.672 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:05 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:30:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T01:30:53.807 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:30:53.808 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-09-15T01:30:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-15T01:30:53.808 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-09-15T01:30:53.808 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:30:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 7.4 M 2024-09-15T01:30:53.808 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-15T01:30:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T01:30:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T01:30:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout:Install 12 Packages 2024-09-15T01:30:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:30:53.810 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 38 M 2024-09-15T01:30:53.810 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 212 M 2024-09-15T01:30:53.810 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:30:53.929 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:11 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:53.965 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T01:30:53.987 INFO:teuthology.orchestra.run.smithi175.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-15T01:30:54.054 INFO:teuthology.orchestra.run.smithi175.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-15T01:30:54.063 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:30:54.064 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:54.064 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-15T01:30:54.064 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:54.065 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:30:54.065 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 47 k 2024-09-15T01:30:54.065 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-15T01:30:54.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T01:30:54.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T01:30:54.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T01:30:54.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T01:30:54.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T01:30:54.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T01:30:54.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T01:30:54.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T01:30:54.065 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T01:30:54.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T01:30:54.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T01:30:54.066 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T01:30:54.066 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:54.066 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:30:54.066 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:54.066 INFO:teuthology.orchestra.run.smithi115.stdout:Install 13 Packages 2024-09-15T01:30:54.066 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:54.067 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 2.2 M 2024-09-15T01:30:54.067 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 26 M 2024-09-15T01:30:54.067 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:30:54.087 INFO:teuthology.orchestra.run.smithi175.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-09-15T01:30:54.121 INFO:teuthology.orchestra.run.smithi175.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 549 kB/s | 18 kB 00:00 2024-09-15T01:30:54.155 INFO:teuthology.orchestra.run.smithi175.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 1.1 MB/s | 38 kB 00:00 2024-09-15T01:30:54.276 INFO:teuthology.orchestra.run.smithi115.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 197 kB/s | 19 kB 00:00 2024-09-15T01:30:54.301 INFO:teuthology.orchestra.run.smithi115.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 209 kB/s | 26 kB 00:00 2024-09-15T01:30:54.322 INFO:teuthology.orchestra.run.smithi175.stdout:(6/12): ceph-mgr-diskprediction-local-17.2.7-16 16 MB/s | 7.4 MB 00:00 2024-09-15T01:30:54.376 INFO:teuthology.orchestra.run.smithi115.stdout:(3/13): ceph-mgr-rook-17.2.7-1643.gd75f2382.el9 241 kB/s | 47 kB 00:00 2024-09-15T01:30:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.6 MB/s | 3.0 MB 00:00 2024-09-15T01:30:54.401 INFO:teuthology.orchestra.run.smithi115.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 222 kB 00:00 2024-09-15T01:30:54.435 INFO:teuthology.orchestra.run.smithi115.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-15T01:30:54.477 INFO:teuthology.orchestra.run.smithi115.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 713 kB/s | 54 kB 00:00 2024-09-15T01:30:54.502 INFO:teuthology.orchestra.run.smithi115.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-15T01:30:54.536 INFO:teuthology.orchestra.run.smithi115.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 8.0 MB/s | 220 kB 00:00 2024-09-15T01:30:54.561 INFO:teuthology.orchestra.run.smithi115.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-09-15T01:30:54.581 INFO:teuthology.orchestra.run.smithi175.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-09-15T01:30:54.586 INFO:teuthology.orchestra.run.smithi115.stdout:(10/13): python3-certifi-2023.05.07-4.el9.noarc 129 kB/s | 14 kB 00:00 2024-09-15T01:30:54.606 INFO:teuthology.orchestra.run.smithi175.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 863 kB/s | 245 kB 00:00 2024-09-15T01:30:54.620 INFO:teuthology.orchestra.run.smithi115.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.3 MB/s | 90 kB 00:00 2024-09-15T01:30:54.631 INFO:teuthology.orchestra.run.smithi175.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 4.9 MB/s | 250 kB 00:00 2024-09-15T01:30:54.661 INFO:teuthology.orchestra.run.smithi115.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 8.2 MB/s | 1.0 MB 00:00 2024-09-15T01:30:54.687 INFO:teuthology.orchestra.run.smithi115.stdout:(13/13): python3-rsa-4.9-2.el9.noarch.rpm 471 kB/s | 59 kB 00:00 2024-09-15T01:30:54.690 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:54.690 INFO:teuthology.orchestra.run.smithi115.stdout:Total 3.5 MB/s | 2.2 MB 00:00 2024-09-15T01:30:54.698 INFO:teuthology.orchestra.run.smithi175.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 9.1 MB/s | 4.9 MB 00:00 2024-09-15T01:30:54.752 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:30:54.767 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:30:54.767 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:30:54.854 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:30:54.854 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:30:55.011 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T01:30:55.041 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-15T01:30:55.044 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T01:30:55.374 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:30:55.541 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T01:30:55.605 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T01:30:55.660 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T01:30:55.697 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T01:30:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T01:30:55.748 INFO:teuthology.orchestra.run.smithi175.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-09-15T01:30:55.749 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:55.749 INFO:teuthology.orchestra.run.smithi175.stdout:Total 20 MB/s | 38 MB 00:01 2024-09-15T01:30:55.840 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T01:30:55.942 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T01:30:55.990 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T01:30:56.099 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:30:56.124 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:30:56.125 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:30:56.149 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T01:30:56.183 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-15T01:30:56.643 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:30:56.643 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:30:56.704 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T01:30:56.763 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T01:30:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T01:30:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T01:30:56.858 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-15T01:30:57.397 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:30:57.476 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T01:30:57.530 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T01:30:57.592 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-15T01:30:57.592 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 1/12 2024-09-15T01:30:57.592 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T01:30:57.592 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T01:30:57.592 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T01:30:57.592 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T01:30:57.592 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T01:30:57.592 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T01:30:57.592 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T01:30:57.592 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T01:30:57.592 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T01:30:57.592 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T01:30:57.630 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T01:30:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T01:30:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-15T01:30:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 1/13 2024-09-15T01:30:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T01:30:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T01:30:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T01:30:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T01:30:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T01:30:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T01:30:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T01:30:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T01:30:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T01:30:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T01:30:57.680 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T01:30:57.882 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T01:30:57.913 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T01:30:57.973 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T01:30:57.974 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T01:30:57.975 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T01:30:57.975 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:30:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T01:30:58.019 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T01:30:58.019 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:58.020 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:30:58.020 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:58.020 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T01:30:58.020 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T01:30:58.020 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T01:30:58.020 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T01:30:58.020 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T01:30:58.020 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T01:30:58.020 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T01:30:58.020 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T01:30:58.020 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T01:30:58.020 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T01:30:58.021 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T01:30:58.021 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T01:30:58.021 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:58.021 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:30:58.295 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T01:30:58.336 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-rook 2024-09-15T01:30:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T01:30:58.748 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-15T01:30:58.748 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 1/12 2024-09-15T01:30:58.748 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T01:30:58.748 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T01:30:58.748 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T01:30:58.748 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T01:30:58.748 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T01:30:58.749 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T01:30:58.749 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T01:30:58.749 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T01:30:58.749 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T01:30:58.749 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T01:30:58.759 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T01:30:58.868 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:16 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:58.910 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:16 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:30:59.001 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:30:59.002 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:59.002 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-15T01:30:59.002 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:59.002 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:30:59.002 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 124 k 2024-09-15T01:30:59.003 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-15T01:30:59.003 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T01:30:59.003 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T01:30:59.003 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T01:30:59.003 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T01:30:59.003 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T01:30:59.003 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:59.003 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:30:59.003 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:30:59.003 INFO:teuthology.orchestra.run.smithi115.stdout:Install 6 Packages 2024-09-15T01:30:59.003 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:30:59.004 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 7.0 M 2024-09-15T01:30:59.004 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 33 M 2024-09-15T01:30:59.004 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:30:59.043 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:30:59.044 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:59.044 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-09-15T01:30:59.044 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:59.044 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 47 k 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T01:30:59.045 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T01:30:59.046 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:59.046 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:30:59.046 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:30:59.046 INFO:teuthology.orchestra.run.smithi152.stdout:Install 13 Packages 2024-09-15T01:30:59.046 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:30:59.046 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 2.2 M 2024-09-15T01:30:59.046 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 26 M 2024-09-15T01:30:59.046 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:30:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T01:30:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:59.144 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:30:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T01:30:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T01:30:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T01:30:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T01:30:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T01:30:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T01:30:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T01:30:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T01:30:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T01:30:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T01:30:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T01:30:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:30:59.145 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:30:59.189 INFO:teuthology.orchestra.run.smithi152.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 284 kB/s | 26 kB 00:00 2024-09-15T01:30:59.214 INFO:teuthology.orchestra.run.smithi152.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 166 kB/s | 19 kB 00:00 2024-09-15T01:30:59.281 INFO:teuthology.orchestra.run.smithi115.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-15T01:30:59.307 INFO:teuthology.orchestra.run.smithi152.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-15T01:30:59.332 INFO:teuthology.orchestra.run.smithi152.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-15T01:30:59.339 INFO:teuthology.orchestra.run.smithi115.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-15T01:30:59.356 INFO:teuthology.orchestra.run.smithi115.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.4 MB/s | 58 kB 00:00 2024-09-15T01:30:59.373 INFO:teuthology.orchestra.run.smithi115.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 4.4 MB/s | 74 kB 00:00 2024-09-15T01:30:59.382 INFO:teuthology.orchestra.run.smithi152.stdout:(5/13): python3-requests-oauthlib-1.3.0-12.el9. 1.1 MB/s | 54 kB 00:00 2024-09-15T01:30:59.424 INFO:teuthology.orchestra.run.smithi152.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 775 kB/s | 32 kB 00:00 2024-09-15T01:30:59.441 INFO:teuthology.orchestra.run.smithi152.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 848 kB/s | 14 kB 00:00 2024-09-15T01:30:59.475 INFO:teuthology.orchestra.run.smithi152.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 6.5 MB/s | 220 kB 00:00 2024-09-15T01:30:59.517 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-rook 2024-09-15T01:30:59.517 INFO:teuthology.orchestra.run.smithi152.stdout:(9/13): python3-kubernetes-26.1.0-2.el9.noarch. 25 MB/s | 1.0 MB 00:00 2024-09-15T01:30:59.532 INFO:teuthology.orchestra.run.smithi115.stdout:(5/6): ceph-mgr-cephadm-17.2.7-1643.gd75f2382.e 274 kB/s | 124 kB 00:00 2024-09-15T01:30:59.533 INFO:teuthology.orchestra.run.smithi152.stdout:(10/13): ceph-mgr-rook-17.2.7-1643.gd75f2382.el 109 kB/s | 47 kB 00:00 2024-09-15T01:30:59.559 INFO:teuthology.orchestra.run.smithi152.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.1 MB/s | 279 kB 00:00 2024-09-15T01:30:59.592 INFO:teuthology.orchestra.run.smithi152.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-09-15T01:30:59.617 INFO:teuthology.orchestra.run.smithi152.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-09-15T01:30:59.620 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:59.620 INFO:teuthology.orchestra.run.smithi152.stdout:Total 3.9 MB/s | 2.2 MB 00:00 2024-09-15T01:30:59.688 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:30:59.704 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:30:59.704 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:30:59.795 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:30:59.796 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:31:00.005 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T01:31:00.116 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:16 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:00.257 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:00.258 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:00.258 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-15T01:31:00.258 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:00.258 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:31:00.258 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 47 k 2024-09-15T01:31:00.258 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-15T01:31:00.258 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:31:00.259 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:00.260 INFO:teuthology.orchestra.run.smithi042.stdout:Install 13 Packages 2024-09-15T01:31:00.260 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:00.260 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 2.2 M 2024-09-15T01:31:00.260 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 26 M 2024-09-15T01:31:00.260 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:31:00.306 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:31:00.432 INFO:teuthology.orchestra.run.smithi115.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.4 MB/s | 6.0 MB 00:01 2024-09-15T01:31:00.434 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:00.434 INFO:teuthology.orchestra.run.smithi115.stdout:Total 4.9 MB/s | 7.0 MB 00:01 2024-09-15T01:31:00.472 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T01:31:00.506 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:31:00.517 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:31:00.517 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:31:00.526 INFO:teuthology.orchestra.run.smithi042.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 259 kB/s | 26 kB 00:00 2024-09-15T01:31:00.527 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T01:31:00.551 INFO:teuthology.orchestra.run.smithi042.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 154 kB/s | 19 kB 00:00 2024-09-15T01:31:00.591 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T01:31:00.615 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:31:00.616 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:31:00.628 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T01:31:00.670 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T01:31:00.677 INFO:teuthology.orchestra.run.smithi042.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-15T01:31:00.710 INFO:teuthology.orchestra.run.smithi042.stdout:(4/13): ceph-mgr-rook-17.2.7-1643.gd75f2382.el9 168 kB/s | 47 kB 00:00 2024-09-15T01:31:00.735 INFO:teuthology.orchestra.run.smithi042.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.2 MB/s | 222 kB 00:00 2024-09-15T01:31:00.777 INFO:teuthology.orchestra.run.smithi042.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 775 kB/s | 32 kB 00:00 2024-09-15T01:31:00.780 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T01:31:00.802 INFO:teuthology.orchestra.run.smithi042.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 571 kB/s | 14 kB 00:00 2024-09-15T01:31:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 8.5 MB/s | 220 kB 00:00 2024-09-15T01:31:00.845 INFO:teuthology.orchestra.run.smithi042.stdout:(9/13): python3-requests-oauthlib-1.3.0-12.el9. 399 kB/s | 54 kB 00:00 2024-09-15T01:31:00.882 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T01:31:00.887 INFO:teuthology.orchestra.run.smithi042.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.3 MB/s | 279 kB 00:00 2024-09-15T01:31:00.912 INFO:teuthology.orchestra.run.smithi042.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 881 kB/s | 59 kB 00:00 2024-09-15T01:31:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T01:31:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:31:00.946 INFO:teuthology.orchestra.run.smithi042.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 8.7 MB/s | 1.0 MB 00:00 2024-09-15T01:31:00.971 INFO:teuthology.orchestra.run.smithi042.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 1.1 MB/s | 90 kB 00:00 2024-09-15T01:31:00.973 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:00.973 INFO:teuthology.orchestra.run.smithi042.stdout:Total 3.1 MB/s | 2.2 MB 00:00 2024-09-15T01:31:01.040 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:31:01.044 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T01:31:01.055 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:31:01.056 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:31:01.086 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-15T01:31:01.099 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T01:31:01.151 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:31:01.153 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:31:01.159 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T01:31:01.650 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T01:31:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:31:01.671 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T01:31:01.718 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T01:31:01.752 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T01:31:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T01:31:01.798 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T01:31:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T01:31:01.821 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-15T01:31:01.846 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-15T01:31:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T01:31:01.916 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T01:31:01.979 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T01:31:02.015 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T01:31:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T01:31:02.163 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T01:31:02.274 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T01:31:02.326 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T01:31:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-15T01:31:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/6 2024-09-15T01:31:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T01:31:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T01:31:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T01:31:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T01:31:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-15T01:31:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 1/13 2024-09-15T01:31:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T01:31:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T01:31:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T01:31:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T01:31:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T01:31:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T01:31:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T01:31:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T01:31:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T01:31:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T01:31:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T01:31:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T01:31:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:02.883 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:31:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T01:31:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T01:31:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T01:31:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T01:31:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T01:31:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:02.884 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:02.952 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T01:31:02.952 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:02.952 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:31:02.952 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:02.952 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:02.953 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:03.097 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T01:31:03.137 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-fuse 2024-09-15T01:31:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T01:31:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T01:31:03.196 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T01:31:03.227 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T01:31:03.259 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-15T01:31:03.582 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-15T01:31:03.582 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 1/12 2024-09-15T01:31:03.582 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T01:31:03.582 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T01:31:03.582 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T01:31:03.582 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T01:31:03.582 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T01:31:03.582 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T01:31:03.582 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T01:31:03.582 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T01:31:03.583 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T01:31:03.583 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T01:31:03.709 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:21 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:03.763 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:21 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:03.842 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:03.843 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-15T01:31:03.843 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:03.843 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:31:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 813 k 2024-09-15T01:31:03.843 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-15T01:31:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T01:31:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:03.844 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:31:03.844 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:03.844 INFO:teuthology.orchestra.run.smithi115.stdout:Install 2 Packages 2024-09-15T01:31:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:03.844 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 893 k 2024-09-15T01:31:03.844 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 2.6 M 2024-09-15T01:31:03.845 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:31:03.897 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:03.897 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 124 k 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:03.899 INFO:teuthology.orchestra.run.smithi152.stdout:Install 6 Packages 2024-09-15T01:31:03.899 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:03.899 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 7.0 M 2024-09-15T01:31:03.899 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 33 M 2024-09-15T01:31:03.899 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:31:03.983 INFO:teuthology.orchestra.run.smithi115.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 684 kB/s | 80 kB 00:00 2024-09-15T01:31:04.021 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T01:31:04.021 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:04.021 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:31:04.021 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:04.021 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T01:31:04.022 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T01:31:04.022 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T01:31:04.022 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T01:31:04.022 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T01:31:04.022 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T01:31:04.022 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T01:31:04.022 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T01:31:04.022 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T01:31:04.022 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T01:31:04.022 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T01:31:04.022 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:04.022 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-15T01:31:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 1/13 2024-09-15T01:31:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T01:31:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T01:31:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T01:31:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T01:31:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T01:31:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T01:31:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T01:31:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T01:31:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T01:31:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T01:31:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T01:31:04.142 INFO:teuthology.orchestra.run.smithi115.stdout:(2/2): ceph-fuse-17.2.7-1643.gd75f2382.el9.x86_ 2.9 MB/s | 813 kB 00:00 2024-09-15T01:31:04.142 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:04.142 INFO:teuthology.orchestra.run.smithi115.stdout:Total 2.9 MB/s | 893 kB 00:00 2024-09-15T01:31:04.154 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:31:04.175 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:31:04.175 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:31:04.235 INFO:teuthology.orchestra.run.smithi152.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.9 MB/s | 248 kB 00:00 2024-09-15T01:31:04.254 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:31:04.254 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:31:04.285 INFO:teuthology.orchestra.run.smithi152.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-15T01:31:04.310 INFO:teuthology.orchestra.run.smithi152.stdout:(3/6): ceph-mgr-cephadm-17.2.7-1643.gd75f2382.e 616 kB/s | 124 kB 00:00 2024-09-15T01:31:04.327 INFO:teuthology.orchestra.run.smithi152.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.3 MB/s | 58 kB 00:00 2024-09-15T01:31:04.386 INFO:teuthology.orchestra.run.smithi152.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 985 kB/s | 74 kB 00:00 2024-09-15T01:31:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:31:04.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-rook 2024-09-15T01:31:04.534 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T01:31:04.534 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:04.534 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:31:04.534 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:04.534 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T01:31:04.534 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T01:31:04.534 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T01:31:04.534 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T01:31:04.535 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T01:31:04.535 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T01:31:04.535 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T01:31:04.535 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T01:31:04.535 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T01:31:04.535 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T01:31:04.535 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T01:31:04.535 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T01:31:04.535 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:04.535 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:04.536 INFO:teuthology.orchestra.run.smithi152.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-15T01:31:04.539 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:04.539 INFO:teuthology.orchestra.run.smithi152.stdout:Total 11 MB/s | 7.0 MB 00:00 2024-09-15T01:31:04.590 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T01:31:04.610 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:31:04.622 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:31:04.622 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:31:04.657 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-15T01:31:04.720 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:31:04.721 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:31:04.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T01:31:05.027 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:31:05.042 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:17 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:05.179 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:05.234 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T01:31:05.235 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:05.235 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-15T01:31:05.235 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:05.235 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:31:05.235 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 47 k 2024-09-15T01:31:05.236 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-15T01:31:05.236 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T01:31:05.236 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T01:31:05.236 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T01:31:05.236 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T01:31:05.236 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T01:31:05.236 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T01:31:05.236 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T01:31:05.236 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T01:31:05.236 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T01:31:05.236 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T01:31:05.237 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T01:31:05.237 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T01:31:05.237 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:05.237 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:31:05.237 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:05.237 INFO:teuthology.orchestra.run.smithi175.stdout:Install 13 Packages 2024-09-15T01:31:05.237 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:05.237 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 2.2 M 2024-09-15T01:31:05.237 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 26 M 2024-09-15T01:31:05.237 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:31:05.263 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T01:31:05.324 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-15T01:31:05.324 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/2 2024-09-15T01:31:05.383 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:21 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:05.395 INFO:teuthology.orchestra.run.smithi175.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 321 kB/s | 26 kB 00:00 2024-09-15T01:31:05.420 INFO:teuthology.orchestra.run.smithi175.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 182 kB/s | 19 kB 00:00 2024-09-15T01:31:05.479 INFO:teuthology.orchestra.run.smithi175.stdout:(3/13): ceph-mgr-rook-17.2.7-1643.gd75f2382.el9 290 kB/s | 47 kB 00:00 2024-09-15T01:31:05.504 INFO:teuthology.orchestra.run.smithi175.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-15T01:31:05.521 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:05.522 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:05.522 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-15T01:31:05.522 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:05.522 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:31:05.522 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 124 k 2024-09-15T01:31:05.522 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-15T01:31:05.522 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T01:31:05.523 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T01:31:05.523 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T01:31:05.523 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T01:31:05.523 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T01:31:05.523 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:05.523 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:31:05.523 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:05.523 INFO:teuthology.orchestra.run.smithi042.stdout:Install 6 Packages 2024-09-15T01:31:05.523 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:05.524 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 7.0 M 2024-09-15T01:31:05.524 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 33 M 2024-09-15T01:31:05.524 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:31:05.554 INFO:teuthology.orchestra.run.smithi175.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-15T01:31:05.580 INFO:teuthology.orchestra.run.smithi175.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 713 kB/s | 54 kB 00:00 2024-09-15T01:31:05.608 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T01:31:05.609 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:05.609 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:31:05.609 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T01:31:05.609 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:05.609 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:05.621 INFO:teuthology.orchestra.run.smithi175.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-15T01:31:05.647 INFO:teuthology.orchestra.run.smithi175.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 210 kB/s | 14 kB 00:00 2024-09-15T01:31:05.672 INFO:teuthology.orchestra.run.smithi175.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-15T01:31:05.697 INFO:teuthology.orchestra.run.smithi175.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.9 MB/s | 220 kB 00:00 2024-09-15T01:31:05.731 INFO:teuthology.orchestra.run.smithi175.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.3 MB/s | 90 kB 00:00 2024-09-15T01:31:05.756 INFO:teuthology.orchestra.run.smithi175.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 706 kB/s | 59 kB 00:00 2024-09-15T01:31:05.781 INFO:teuthology.orchestra.run.smithi042.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.9 MB/s | 248 kB 00:00 2024-09-15T01:31:05.789 INFO:teuthology.orchestra.run.smithi175.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-15T01:31:05.792 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:05.792 INFO:teuthology.orchestra.run.smithi175.stdout:Total 3.6 MB/s | 2.2 MB 00:00 2024-09-15T01:31:05.803 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T01:31:05.831 INFO:teuthology.orchestra.run.smithi042.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-15T01:31:05.854 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:31:05.865 INFO:teuthology.orchestra.run.smithi042.stdout:(3/6): ceph-mgr-cephadm-17.2.7-1643.gd75f2382.e 591 kB/s | 124 kB 00:00 2024-09-15T01:31:05.869 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:31:05.869 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:31:05.890 INFO:teuthology.orchestra.run.smithi042.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 984 kB/s | 58 kB 00:00 2024-09-15T01:31:05.890 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T01:31:05.932 INFO:teuthology.orchestra.run.smithi042.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-15T01:31:05.939 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-volume 2024-09-15T01:31:05.946 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T01:31:05.957 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:31:05.959 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:31:05.974 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-15T01:31:06.024 INFO:teuthology.orchestra.run.smithi042.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-15T01:31:06.026 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:06.027 INFO:teuthology.orchestra.run.smithi042.stdout:Total 14 MB/s | 7.0 MB 00:00 2024-09-15T01:31:06.099 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:31:06.111 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:31:06.111 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:31:06.215 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:31:06.215 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:31:06.459 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:31:06.518 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:24 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:06.524 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:31:06.629 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T01:31:06.653 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:06.654 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:06.654 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-15T01:31:06.654 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:06.654 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:31:06.654 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 270 k 2024-09-15T01:31:06.655 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:06.655 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:31:06.655 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:06.655 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-15T01:31:06.655 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:06.655 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 270 k 2024-09-15T01:31:06.655 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 1.3 M 2024-09-15T01:31:06.655 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:31:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T01:31:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T01:31:06.737 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T01:31:06.761 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T01:31:06.775 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T01:31:06.776 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-15T01:31:06.776 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/6 2024-09-15T01:31:06.776 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T01:31:06.776 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T01:31:06.776 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T01:31:06.776 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T01:31:06.816 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T01:31:06.888 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-volume-17.2.7-1643.gd75f2382.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-15T01:31:06.889 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:06.889 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-15T01:31:06.889 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:31:06.897 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:31:06.897 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:31:06.926 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:31:06.926 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:31:06.927 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T01:31:07.027 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T01:31:07.052 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T01:31:07.052 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:07.052 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:31:07.052 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:07.053 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T01:31:07.053 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T01:31:07.053 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T01:31:07.053 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T01:31:07.053 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T01:31:07.053 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:07.053 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:07.076 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T01:31:07.141 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:31:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:07.222 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:31:07.223 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T01:31:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:07.275 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-fuse 2024-09-15T01:31:07.325 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T01:31:07.416 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T01:31:07.478 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T01:31:07.505 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-15T01:31:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T01:31:07.836 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:25 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T01:31:07.882 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T01:31:07.928 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T01:31:07.952 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-15T01:31:07.967 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:07.968 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:07.968 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-09-15T01:31:07.968 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:07.968 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:31:07.968 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 813 k 2024-09-15T01:31:07.968 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-09-15T01:31:07.968 INFO:teuthology.orchestra.run.smithi152.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T01:31:07.968 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:07.968 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:31:07.968 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:07.969 INFO:teuthology.orchestra.run.smithi152.stdout:Install 2 Packages 2024-09-15T01:31:07.969 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:07.969 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 893 k 2024-09-15T01:31:07.969 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 2.6 M 2024-09-15T01:31:07.969 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:31:08.145 INFO:teuthology.orchestra.run.smithi152.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 524 kB/s | 80 kB 00:00 2024-09-15T01:31:08.245 INFO:teuthology.orchestra.run.smithi152.stdout:(2/2): ceph-fuse-17.2.7-1643.gd75f2382.el9.x86_ 3.1 MB/s | 813 kB 00:00 2024-09-15T01:31:08.246 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:08.246 INFO:teuthology.orchestra.run.smithi152.stdout:Total 3.1 MB/s | 893 kB 00:00 2024-09-15T01:31:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:08.249 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:31:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:08.250 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:31:08.278 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:31:08.278 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:31:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-15T01:31:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/6 2024-09-15T01:31:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T01:31:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T01:31:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T01:31:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T01:31:08.358 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:31:08.358 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:31:08.545 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:31:08.550 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados-devel 2024-09-15T01:31:08.557 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T01:31:08.558 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:08.558 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:31:08.558 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:08.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T01:31:08.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T01:31:08.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T01:31:08.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T01:31:08.558 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T01:31:08.558 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:08.558 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:08.670 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T01:31:08.735 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-15T01:31:08.750 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-15T01:31:08.750 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 1/13 2024-09-15T01:31:08.750 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T01:31:08.750 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T01:31:08.751 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T01:31:08.751 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T01:31:08.751 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T01:31:08.751 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T01:31:08.751 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T01:31:08.751 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T01:31:08.751 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T01:31:08.751 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T01:31:08.751 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T01:31:08.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-fuse 2024-09-15T01:31:09.077 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T01:31:09.078 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:09.078 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:31:09.078 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:09.078 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T01:31:09.078 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T01:31:09.078 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T01:31:09.078 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T01:31:09.079 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T01:31:09.079 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T01:31:09.079 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T01:31:09.079 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T01:31:09.079 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T01:31:09.079 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T01:31:09.079 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T01:31:09.079 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T01:31:09.079 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:09.079 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:09.124 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:27 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:09.261 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:09.261 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:09.261 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-15T01:31:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:31:09.262 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 126 k 2024-09-15T01:31:09.262 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:31:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-15T01:31:09.262 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 126 k 2024-09-15T01:31:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 448 k 2024-09-15T01:31:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:31:09.324 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T01:31:09.363 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-15T01:31:09.363 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/2 2024-09-15T01:31:09.395 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:25 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:09.476 INFO:teuthology.orchestra.run.smithi115.stdout:librados-devel-17.2.7-1643.gd75f2382.el9.x86_64 591 kB/s | 126 kB 00:00 2024-09-15T01:31:09.477 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:09.477 INFO:teuthology.orchestra.run.smithi115.stdout:Total 586 kB/s | 126 kB 00:00 2024-09-15T01:31:09.478 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:31:09.483 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:31:09.483 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:31:09.534 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:09.534 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-15T01:31:09.534 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:09.534 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:31:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 813 k 2024-09-15T01:31:09.535 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-15T01:31:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T01:31:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:09.535 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:31:09.535 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:09.535 INFO:teuthology.orchestra.run.smithi042.stdout:Install 2 Packages 2024-09-15T01:31:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:09.536 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 893 k 2024-09-15T01:31:09.536 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 2.6 M 2024-09-15T01:31:09.536 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:31:09.546 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:31:09.547 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:31:09.601 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T01:31:09.601 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:31:09.601 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T01:31:09.601 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:09.601 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:09.697 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:31:09.767 INFO:teuthology.orchestra.run.smithi042.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 435 kB/s | 80 kB 00:00 2024-09-15T01:31:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:(2/2): ceph-fuse-17.2.7-1643.gd75f2382.el9.x86_ 3.2 MB/s | 813 kB 00:00 2024-09-15T01:31:09.834 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:09.835 INFO:teuthology.orchestra.run.smithi042.stdout:Total 2.9 MB/s | 893 kB 00:00 2024-09-15T01:31:09.847 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:31:09.865 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-volume 2024-09-15T01:31:09.869 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:31:09.869 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:31:09.889 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:21 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:09.952 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:31:09.953 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:31:10.023 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:10.024 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:10.024 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-15T01:31:10.024 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:10.024 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:31:10.024 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 124 k 2024-09-15T01:31:10.024 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-15T01:31:10.024 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T01:31:10.024 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T01:31:10.024 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T01:31:10.025 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T01:31:10.025 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T01:31:10.025 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:10.025 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:31:10.025 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:10.025 INFO:teuthology.orchestra.run.smithi175.stdout:Install 6 Packages 2024-09-15T01:31:10.025 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:10.026 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 7.0 M 2024-09-15T01:31:10.026 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 33 M 2024-09-15T01:31:10.026 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:31:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:31:10.250 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T01:31:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-15T01:31:10.391 INFO:teuthology.orchestra.run.smithi175.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1643.gd75f2382.e 659 kB/s | 124 kB 00:00 2024-09-15T01:31:10.416 INFO:teuthology.orchestra.run.smithi175.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-15T01:31:10.450 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:28 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:10.466 INFO:teuthology.orchestra.run.smithi175.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-15T01:31:10.491 INFO:teuthology.orchestra.run.smithi175.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 770 kB/s | 58 kB 00:00 2024-09-15T01:31:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:10.506 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:31:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:10.506 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:10.517 INFO:teuthology.orchestra.run.smithi175.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.5 MB/s | 74 kB 00:00 2024-09-15T01:31:10.588 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:10.589 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:10.589 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-09-15T01:31:10.589 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:10.589 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:31:10.589 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 270 k 2024-09-15T01:31:10.590 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:10.590 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:31:10.590 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:10.590 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-09-15T01:31:10.590 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:10.590 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 270 k 2024-09-15T01:31:10.590 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 1.3 M 2024-09-15T01:31:10.590 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:31:10.728 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs2 2024-09-15T01:31:10.816 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-volume-17.2.7-1643.gd75f2382.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-15T01:31:10.816 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:10.817 INFO:teuthology.orchestra.run.smithi152.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-15T01:31:10.817 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:31:10.825 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:31:10.825 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:31:10.855 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:31:10.856 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:31:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-15T01:31:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/2 2024-09-15T01:31:11.064 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T01:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:11.310 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:29 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:11.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T01:31:11.343 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:11.343 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:31:11.343 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T01:31:11.343 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:11.343 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:11.402 INFO:teuthology.orchestra.run.smithi115.stdout:Package libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:11.448 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:11.448 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-15T01:31:11.448 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:11.517 INFO:teuthology.orchestra.run.smithi175.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.5 MB/s | 6.0 MB 00:01 2024-09-15T01:31:11.519 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:11.519 INFO:teuthology.orchestra.run.smithi175.stdout:Total 4.7 MB/s | 7.0 MB 00:01 2024-09-15T01:31:11.521 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs-devel 2024-09-15T01:31:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:31:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:31:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:31:11.698 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-volume 2024-09-15T01:31:11.700 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:31:11.700 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:31:12.005 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:31:12.089 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:30 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:12.145 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:31:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:12.145 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:12.185 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T01:31:12.219 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:12.220 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-15T01:31:12.220 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:12.220 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:31:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-15T01:31:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:12.224 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:31:12.224 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:12.224 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-15T01:31:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:12.224 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 25 k 2024-09-15T01:31:12.224 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 110 k 2024-09-15T01:31:12.224 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:31:12.253 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T01:31:12.291 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:28 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout:libcephfs-devel-17.2.7-1643.gd75f2382.el9.x86_6 175 kB/s | 25 kB 00:00 2024-09-15T01:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout:Total 174 kB/s | 25 kB 00:00 2024-09-15T01:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:31:12.430 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install librados-devel 2024-09-15T01:31:12.432 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:12.432 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-15T01:31:12.432 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:12.432 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:31:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 270 k 2024-09-15T01:31:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:12.432 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:31:12.432 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:12.432 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-15T01:31:12.432 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:12.433 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 270 k 2024-09-15T01:31:12.433 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 1.3 M 2024-09-15T01:31:12.433 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:12.672 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-volume-17.2.7-1643.gd75f2382.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-15T01:31:12.673 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:12.673 INFO:teuthology.orchestra.run.smithi042.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-15T01:31:12.673 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:31:12.682 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:31:12.682 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:31:12.717 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:31:12.718 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:31:12.783 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T01:31:12.872 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T01:31:12.936 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T01:31:12.936 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:12.938 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:31:12.964 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-15T01:31:12.999 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:13.015 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:13.016 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:31:13.016 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T01:31:13.016 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:13.017 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:31 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:13.154 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:13.154 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:13.154 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2024-09-15T01:31:13.154 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:13.154 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:31:13.154 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 126 k 2024-09-15T01:31:13.155 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:13.155 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:31:13.155 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:13.155 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-09-15T01:31:13.155 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:13.155 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 126 k 2024-09-15T01:31:13.155 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 448 k 2024-09-15T01:31:13.155 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:31:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:31:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:13.346 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados2 2024-09-15T01:31:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-15T01:31:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/6 2024-09-15T01:31:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T01:31:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T01:31:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T01:31:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T01:31:13.908 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:31 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:13.997 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T01:31:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:14.001 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:31:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T01:31:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T01:31:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T01:31:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T01:31:14.001 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T01:31:14.002 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:14.002 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:14.025 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:31:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:14.026 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:14.040 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:14.041 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-15T01:31:14.041 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:14.109 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librbd1 2024-09-15T01:31:14.256 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-fuse 2024-09-15T01:31:14.313 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librados-devel 2024-09-15T01:31:14.670 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:32 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:14.761 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:14.804 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:14.805 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-15T01:31:14.805 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:14.826 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:26 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:14.879 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rados 2024-09-15T01:31:14.912 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:30 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:14.959 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:14.960 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:14.960 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-15T01:31:14.960 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:14.960 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:31:14.960 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 813 k 2024-09-15T01:31:14.960 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-15T01:31:14.961 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T01:31:14.961 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:14.961 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:31:14.961 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:14.961 INFO:teuthology.orchestra.run.smithi175.stdout:Install 2 Packages 2024-09-15T01:31:14.961 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:14.961 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 893 k 2024-09-15T01:31:14.962 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 2.6 M 2024-09-15T01:31:14.962 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:31:15.051 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:15.051 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:15.052 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-09-15T01:31:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:31:15.052 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 126 k 2024-09-15T01:31:15.052 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:31:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-15T01:31:15.052 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 126 k 2024-09-15T01:31:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 448 k 2024-09-15T01:31:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:31:15.269 INFO:teuthology.orchestra.run.smithi042.stdout:librados-devel-17.2.7-1643.gd75f2382.el9.x86_64 586 kB/s | 126 kB 00:00 2024-09-15T01:31:15.269 INFO:teuthology.orchestra.run.smithi175.stdout:(1/2): ceph-fuse-17.2.7-1643.gd75f2382.el9.x86_ 3.0 MB/s | 813 kB 00:00 2024-09-15T01:31:15.270 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:15.270 INFO:teuthology.orchestra.run.smithi042.stdout:Total 581 kB/s | 126 kB 00:00 2024-09-15T01:31:15.270 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:31:15.275 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:31:15.276 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:31:15.342 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:31:15.342 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:31:15.385 INFO:teuthology.orchestra.run.smithi175.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 211 kB/s | 80 kB 00:00 2024-09-15T01:31:15.386 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:15.386 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.0 MB/s | 893 kB 00:00 2024-09-15T01:31:15.398 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:31:15.418 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:31:15.418 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:31:15.440 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:33 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:15.495 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:31:15.498 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:31:15.498 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:31:15.529 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:15.553 INFO:teuthology.orchestra.run.smithi152.stdout:librados-devel-17.2.7-1643.gd75f2382.el9.x86_64 53 kB/s | 126 kB 00:02 2024-09-15T01:31:15.554 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:15.554 INFO:teuthology.orchestra.run.smithi152.stdout:Total 53 kB/s | 126 kB 00:02 2024-09-15T01:31:15.554 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:31:15.561 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:31:15.561 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:31:15.571 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:15.572 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-15T01:31:15.572 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:15.594 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:15.629 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:31:15.629 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:31:15.653 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rgw 2024-09-15T01:31:15.688 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:31:15.787 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:31:15.842 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T01:31:15.891 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:15.909 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-15T01:31:16.021 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:16.219 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:34 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:16.302 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:16.308 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:16.351 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:16.352 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-15T01:31:16.352 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:16.381 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:16.382 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:16.382 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:31:16.382 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:16.382 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:16.382 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:16.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-cephfs 2024-09-15T01:31:16.551 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:16.551 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:16.551 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:31:16.551 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:16.551 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:16.551 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:16.576 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-15T01:31:16.576 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/2 2024-09-15T01:31:16.631 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install libcephfs2 2024-09-15T01:31:16.773 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install libcephfs2 2024-09-15T01:31:16.879 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T01:31:16.880 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:16.880 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:31:16.880 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T01:31:16.880 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:16.880 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:16.981 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:34 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:17.070 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:17.112 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:17.113 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-15T01:31:17.113 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:17.147 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-volume 2024-09-15T01:31:17.183 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rbd 2024-09-15T01:31:17.225 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:33 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:17.319 INFO:teuthology.orchestra.run.smithi042.stdout:Package libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:17.348 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:35 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:17.365 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:17.368 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-15T01:31:17.368 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:17.441 INFO:teuthology.orchestra.run.smithi152.stdout:Package libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:17.442 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install libcephfs-devel 2024-09-15T01:31:17.486 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:17.487 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-09-15T01:31:17.487 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:17.551 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install libcephfs-devel 2024-09-15T01:31:17.725 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:29 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:17.741 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:35 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:17.832 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:17.861 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:17.862 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:17.862 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-15T01:31:17.862 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:17.862 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:31:17.862 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 270 k 2024-09-15T01:31:17.862 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:17.862 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:31:17.862 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:17.862 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-15T01:31:17.862 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:17.863 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 270 k 2024-09-15T01:31:17.863 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 1.3 M 2024-09-15T01:31:17.863 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:31:17.874 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:17.875 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-15T01:31:17.875 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:17.941 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-fuse 2024-09-15T01:31:18.034 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:34 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:18.098 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-volume-17.2.7-1643.gd75f2382.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-15T01:31:18.098 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:18.098 INFO:teuthology.orchestra.run.smithi175.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-15T01:31:18.099 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:31:18.107 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:31:18.107 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:31:18.114 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:36 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:18.134 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:31:18.134 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:31:18.171 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:18.172 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-09-15T01:31:18.172 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:18.172 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:31:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-15T01:31:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:18.172 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:31:18.173 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:18.173 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-15T01:31:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:18.173 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 25 k 2024-09-15T01:31:18.173 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 110 k 2024-09-15T01:31:18.173 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:31:18.247 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:18.247 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:18.247 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2024-09-15T01:31:18.248 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:18.248 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:31:18.248 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-15T01:31:18.248 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:18.248 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:31:18.248 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:18.248 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-09-15T01:31:18.248 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:18.249 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 25 k 2024-09-15T01:31:18.249 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 110 k 2024-09-15T01:31:18.249 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:31:18.332 INFO:teuthology.orchestra.run.smithi042.stdout:libcephfs-devel-17.2.7-1643.gd75f2382.el9.x86_6 158 kB/s | 25 kB 00:00 2024-09-15T01:31:18.332 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:18.332 INFO:teuthology.orchestra.run.smithi042.stdout:Total 156 kB/s | 25 kB 00:00 2024-09-15T01:31:18.332 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:31:18.336 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:31:18.336 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:31:18.338 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:31:18.338 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:31:18.341 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:31:18.406 INFO:teuthology.orchestra.run.smithi152.stdout:libcephfs-devel-17.2.7-1643.gd75f2382.el9.x86_6 158 kB/s | 25 kB 00:00 2024-09-15T01:31:18.406 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:18.407 INFO:teuthology.orchestra.run.smithi152.stdout:Total 157 kB/s | 25 kB 00:00 2024-09-15T01:31:18.407 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:31:18.408 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:18.410 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:31:18.410 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:31:18.412 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:31:18.412 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:31:18.424 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:18.424 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:31:18.424 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T01:31:18.424 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:31:18.498 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:31:18.511 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:36 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:18.603 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:18.644 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:18.645 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:18.645 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-15T01:31:18.645 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:18.645 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:31:18.645 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 82 k 2024-09-15T01:31:18.645 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:18.645 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:31:18.646 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:18.646 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-15T01:31:18.646 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:18.646 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 82 k 2024-09-15T01:31:18.646 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 222 k 2024-09-15T01:31:18.646 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:31:18.841 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-fuse-17.2.7-1643.gd75f2382.el9.x86_64.rpm 421 kB/s | 82 kB 00:00 2024-09-15T01:31:18.842 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:18.842 INFO:teuthology.orchestra.run.smithi115.stdout:Total 418 kB/s | 82 kB 00:00 2024-09-15T01:31:18.842 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:31:18.848 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:31:18.848 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:31:18.911 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:31:18.912 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:31:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:18.981 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:19.061 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:31:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:19.164 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:31:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:19.165 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:19.209 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:19.209 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:19.210 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:31:19.210 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:19.210 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:19.210 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:19.356 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librados2 2024-09-15T01:31:19.398 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install librados2 2024-09-15T01:31:19.434 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:19.434 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:19.434 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:31:19.434 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:19.434 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:19.434 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:19.546 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:19.703 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librados-devel 2024-09-15T01:31:19.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:19.797 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:19.797 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:31:19.797 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:19.797 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:19.797 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:19.952 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:35 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:19.967 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:37 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:20.047 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:20.058 INFO:teuthology.orchestra.run.smithi152.stdout:Package librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:20.093 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-mirror 2024-09-15T01:31:20.094 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:20.094 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-15T01:31:20.095 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:20.101 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:20.102 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-09-15T01:31:20.102 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:20.166 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install librbd1 2024-09-15T01:31:20.175 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librbd1 2024-09-15T01:31:20.295 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:32 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:20.428 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2024-09-15T01:31:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:31:20.429 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 126 k 2024-09-15T01:31:20.429 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:20.429 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:31:20.429 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:20.429 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-15T01:31:20.429 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:20.433 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 126 k 2024-09-15T01:31:20.433 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 448 k 2024-09-15T01:31:20.433 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:31:20.611 INFO:teuthology.orchestra.run.smithi175.stdout:librados-devel-17.2.7-1643.gd75f2382.el9.x86_64 696 kB/s | 126 kB 00:00 2024-09-15T01:31:20.611 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:20.612 INFO:teuthology.orchestra.run.smithi175.stdout:Total 690 kB/s | 126 kB 00:00 2024-09-15T01:31:20.612 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:31:20.617 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:31:20.617 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:31:20.666 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:38 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:20.680 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:31:20.681 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:31:20.732 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:38 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:20.771 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:36 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:20.805 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:20.806 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-15T01:31:20.806 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:20.806 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:31:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-15T01:31:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:20.806 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:31:20.806 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:20.806 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-15T01:31:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:20.807 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 2.9 M 2024-09-15T01:31:20.807 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 12 M 2024-09-15T01:31:20.807 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:31:20.822 INFO:teuthology.orchestra.run.smithi152.stdout:Package librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:20.840 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:31:20.865 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:20.865 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:20.866 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-09-15T01:31:20.866 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:20.911 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:20.912 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-15T01:31:20.913 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:20.932 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-rados 2024-09-15T01:31:20.950 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:20.990 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rados 2024-09-15T01:31:21.161 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-mirror-17.2.7-1643.gd75f2382.el9.x86_64.rpm 8.3 MB/s | 2.9 MB 00:00 2024-09-15T01:31:21.162 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:21.162 INFO:teuthology.orchestra.run.smithi115.stdout:Total 8.3 MB/s | 2.9 MB 00:00 2024-09-15T01:31:21.163 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:31:21.171 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:31:21.171 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:31:21.253 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:31:21.253 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:31:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:21.496 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:39 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:21.581 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:37 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:21.586 INFO:teuthology.orchestra.run.smithi152.stdout:Package python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:21.628 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:21.629 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-09-15T01:31:21.630 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:21.658 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:31:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:21.659 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:31:21.674 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:21.695 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-rgw 2024-09-15T01:31:21.720 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:21.721 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-15T01:31:21.721 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:21.753 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:31:21.753 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T01:31:21.754 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T01:31:21.754 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T01:31:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:21.799 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rgw 2024-09-15T01:31:21.878 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install libcephfs2 2024-09-15T01:31:22.264 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:40 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:22.355 INFO:teuthology.orchestra.run.smithi152.stdout:Package python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:22.380 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:38 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:22.398 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:22.399 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-09-15T01:31:22.399 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:22.446 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:34 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:22.469 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-cephfs 2024-09-15T01:31:22.472 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:22.517 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:22.518 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-15T01:31:22.519 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:22.536 INFO:teuthology.orchestra.run.smithi175.stdout:Package libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:22.581 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:22.582 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-15T01:31:22.582 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:22.600 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-cephfs 2024-09-15T01:31:22.643 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install libcephfs-devel 2024-09-15T01:31:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:22.814 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:31:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:22.814 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:23.035 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:41 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:23.046 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-nbd 2024-09-15T01:31:23.126 INFO:teuthology.orchestra.run.smithi152.stdout:Package python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:23.168 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:23.169 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-09-15T01:31:23.170 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:23.190 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:39 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:23.214 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:35 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:23.235 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-rbd 2024-09-15T01:31:23.284 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:23.331 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:23.332 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-15T01:31:23.332 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:23.347 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:23.348 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:23.348 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2024-09-15T01:31:23.349 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:23.349 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:31:23.349 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-15T01:31:23.349 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:23.349 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:31:23.349 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:23.349 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-15T01:31:23.349 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:23.349 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 25 k 2024-09-15T01:31:23.349 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 110 k 2024-09-15T01:31:23.349 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:31:23.406 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rbd 2024-09-15T01:31:23.618 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:41 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:23.630 INFO:teuthology.orchestra.run.smithi175.stdout:libcephfs-devel-17.2.7-1643.gd75f2382.el9.x86_6 89 kB/s | 25 kB 00:00 2024-09-15T01:31:23.630 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:23.631 INFO:teuthology.orchestra.run.smithi175.stdout:Total 88 kB/s | 25 kB 00:00 2024-09-15T01:31:23.631 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:31:23.634 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:31:23.635 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:31:23.636 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:31:23.636 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:31:23.734 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:31:23.754 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-15T01:31:23.755 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:23.755 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-15T01:31:23.755 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:23.755 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-15T01:31:23.755 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 165 k 2024-09-15T01:31:23.755 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:23.755 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-15T01:31:23.756 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-15T01:31:23.756 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-15T01:31:23.756 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:23.756 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 165 k 2024-09-15T01:31:23.756 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 477 k 2024-09-15T01:31:23.756 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-15T01:31:23.801 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:41 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:23.860 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:23.891 INFO:teuthology.orchestra.run.smithi152.stdout:Package python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:23.934 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:23.935 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-09-15T01:31:23.935 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:23.990 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:39 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:23.999 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install rbd-fuse 2024-09-15T01:31:24.082 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:24.128 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:24.129 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-15T01:31:24.129 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:24.131 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-nbd-17.2.7-1643.gd75f2382.el9.x86_64.rpm 440 kB/s | 165 kB 00:00 2024-09-15T01:31:24.132 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:24.132 INFO:teuthology.orchestra.run.smithi115.stdout:Total 439 kB/s | 165 kB 00:00 2024-09-15T01:31:24.132 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-15T01:31:24.140 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-15T01:31:24.140 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-15T01:31:24.203 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-15T01:31:24.203 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-15T01:31:24.216 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-fuse 2024-09-15T01:31:24.220 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-15T01:31:24.458 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:24.458 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:24.458 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:31:24.458 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:24.458 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:24.458 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:24.462 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:24.568 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:42 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:24.647 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librados2 2024-09-15T01:31:24.701 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 82 k 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 82 k 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 222 k 2024-09-15T01:31:24.702 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:31:24.808 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:40 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:24.947 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:24.947 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:24.947 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-15T01:31:24.947 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:24.948 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:31:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 82 k 2024-09-15T01:31:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:24.948 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:31:24.948 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:24.948 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-15T01:31:24.948 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:24.948 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 82 k 2024-09-15T01:31:24.948 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 222 k 2024-09-15T01:31:24.948 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:31:25.076 INFO:teuthology.orchestra.run.smithi152.stdout:rbd-fuse-17.2.7-1643.gd75f2382.el9.x86_64.rpm 220 kB/s | 82 kB 00:00 2024-09-15T01:31:25.076 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:25.077 INFO:teuthology.orchestra.run.smithi152.stdout:Total 219 kB/s | 82 kB 00:00 2024-09-15T01:31:25.077 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:31:25.084 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:31:25.084 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:31:25.146 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:31:25.146 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:31:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:25.178 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-15T01:31:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:25.178 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-15T01:31:25.220 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:37 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-fuse-17.2.7-1643.gd75f2382.el9.x86_64.rpm 244 kB/s | 82 kB 00:00 2024-09-15T01:31:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:Total 243 kB/s | 82 kB 00:00 2024-09-15T01:31:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:31:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:31:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:31:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:31:25.312 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:25.355 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:25.356 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-15T01:31:25.356 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:31:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:31:25.388 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:25.424 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librbd1 2024-09-15T01:31:25.459 DEBUG:teuthology.parallel:result is None 2024-09-15T01:31:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:31:25.627 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:25.765 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:25.992 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:25.995 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:37 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:26.021 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:26.021 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:26.021 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:31:26.021 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:26.021 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:26.021 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:26.085 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:26.128 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:26.129 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-15T01:31:26.129 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:26.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rados 2024-09-15T01:31:26.272 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:26.272 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:26.272 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:31:26.272 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:26.272 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:26.272 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:26.293 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install rbd-mirror 2024-09-15T01:31:26.558 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-mirror 2024-09-15T01:31:26.755 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:38 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:26.846 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:26.873 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:44 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:26.888 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:26.889 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-15T01:31:26.889 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:26.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rgw 2024-09-15T01:31:27.009 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:27.010 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:27.010 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2024-09-15T01:31:27.010 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:27.010 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:31:27.010 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-15T01:31:27.010 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:27.010 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:31:27.010 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:27.011 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-09-15T01:31:27.011 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:27.011 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 2.9 M 2024-09-15T01:31:27.011 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 12 M 2024-09-15T01:31:27.011 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:31:27.157 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:43 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:27.296 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:27.297 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:27.297 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-09-15T01:31:27.297 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:27.297 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:31:27.297 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-15T01:31:27.297 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:27.297 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:31:27.297 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:27.297 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-15T01:31:27.298 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:27.298 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 2.9 M 2024-09-15T01:31:27.298 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 12 M 2024-09-15T01:31:27.298 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:31:27.353 INFO:teuthology.orchestra.run.smithi152.stdout:rbd-mirror-17.2.7-1643.gd75f2382.el9.x86_64.rpm 8.6 MB/s | 2.9 MB 00:00 2024-09-15T01:31:27.355 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:27.355 INFO:teuthology.orchestra.run.smithi152.stdout:Total 8.5 MB/s | 2.9 MB 00:00 2024-09-15T01:31:27.355 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:31:27.364 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:31:27.364 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:31:27.443 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:31:27.444 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:31:27.521 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:39 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:27.610 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:27.638 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-mirror-17.2.7-1643.gd75f2382.el9.x86_64.rpm 8.6 MB/s | 2.9 MB 00:00 2024-09-15T01:31:27.639 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:27.640 INFO:teuthology.orchestra.run.smithi042.stdout:Total 8.6 MB/s | 2.9 MB 00:00 2024-09-15T01:31:27.640 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:31:27.650 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:31:27.650 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:31:27.652 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:27.653 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-15T01:31:27.653 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:27.718 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-cephfs 2024-09-15T01:31:27.735 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:31:27.736 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:31:27.854 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:31:27.921 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:27.945 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:27.946 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:31:27.946 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T01:31:27.946 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T01:31:27.946 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T01:31:27.946 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:31:28.231 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:28.259 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:28.259 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:31:28.259 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T01:31:28.259 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T01:31:28.259 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T01:31:28.259 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:28.290 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:40 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:28.382 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:28.424 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:28.425 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-15T01:31:28.425 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:28.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rbd 2024-09-15T01:31:29.044 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:29.044 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:29.044 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:31:29.045 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:29.045 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:29.045 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:29.056 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:41 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:29.146 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:29.188 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:29.190 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-15T01:31:29.190 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:29.252 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-fuse 2024-09-15T01:31:29.305 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install rbd-nbd 2024-09-15T01:31:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:29.450 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:31:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:29.450 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:29.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-nbd 2024-09-15T01:31:29.820 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:41 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:29.889 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:47 ago on Sun 15 Sep 2024 01:29:42 AM UTC. 2024-09-15T01:31:29.953 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:29.953 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:29.953 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-15T01:31:29.954 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:29.954 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:31:29.954 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 82 k 2024-09-15T01:31:29.954 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:29.954 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:31:29.954 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:29.954 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-15T01:31:29.954 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:29.955 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 82 k 2024-09-15T01:31:29.955 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 222 k 2024-09-15T01:31:29.955 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:31:30.026 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-15T01:31:30.027 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:30.027 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-09-15T01:31:30.027 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:30.027 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-15T01:31:30.027 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 165 k 2024-09-15T01:31:30.027 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:30.027 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-15T01:31:30.027 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-15T01:31:30.028 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-09-15T01:31:30.028 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:30.028 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 165 k 2024-09-15T01:31:30.028 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 477 k 2024-09-15T01:31:30.028 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-15T01:31:30.141 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-fuse-17.2.7-1643.gd75f2382.el9.x86_64.rpm 439 kB/s | 82 kB 00:00 2024-09-15T01:31:30.141 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:30.142 INFO:teuthology.orchestra.run.smithi175.stdout:Total 436 kB/s | 82 kB 00:00 2024-09-15T01:31:30.142 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:31:30.149 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:31:30.149 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:31:30.212 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:31:30.212 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:31:30.231 INFO:teuthology.orchestra.run.smithi152.stdout:rbd-nbd-17.2.7-1643.gd75f2382.el9.x86_64.rpm 812 kB/s | 165 kB 00:00 2024-09-15T01:31:30.232 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:30.232 INFO:teuthology.orchestra.run.smithi152.stdout:Total 805 kB/s | 165 kB 00:00 2024-09-15T01:31:30.232 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-15T01:31:30.240 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-15T01:31:30.240 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-15T01:31:30.304 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-15T01:31:30.304 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-15T01:31:30.322 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:46 ago on Sun 15 Sep 2024 01:29:44 AM UTC. 2024-09-15T01:31:30.358 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:31:30.454 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:30.462 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-15T01:31:30.462 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-15T01:31:30.463 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-15T01:31:30.463 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:30.463 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-15T01:31:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 165 k 2024-09-15T01:31:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:30.463 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-15T01:31:30.464 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-15T01:31:30.464 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-15T01:31:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:30.464 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 165 k 2024-09-15T01:31:30.464 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 477 k 2024-09-15T01:31:30.464 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-15T01:31:30.570 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:30.650 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-nbd-17.2.7-1643.gd75f2382.el9.x86_64.rpm 891 kB/s | 165 kB 00:00 2024-09-15T01:31:30.650 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:30.651 INFO:teuthology.orchestra.run.smithi042.stdout:Total 883 kB/s | 165 kB 00:00 2024-09-15T01:31:30.651 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-15T01:31:30.659 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-15T01:31:30.659 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-15T01:31:30.726 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-15T01:31:30.726 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-15T01:31:30.833 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:31.000 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-15T01:31:31.057 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:31.057 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:31.057 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:31:31.057 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:31.057 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:31.057 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:31.063 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:31.117 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:31.343 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-mirror 2024-09-15T01:31:31.421 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:31.421 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:31.421 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-15T01:31:31.421 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:31.421 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:31.421 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-15T01:31:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:31.687 DEBUG:teuthology.parallel:result is None 2024-09-15T01:31:31.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:31.820 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:31.820 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-15T01:31:31.820 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:31.821 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:31.821 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-15T01:31:31.935 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:43 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:32.073 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:32.073 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2024-09-15T01:31:32.074 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:32.074 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:31:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-15T01:31:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:32.074 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:31:32.074 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:32.074 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-15T01:31:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:32.074 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 2.9 M 2024-09-15T01:31:32.074 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 12 M 2024-09-15T01:31:32.074 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:31:32.097 DEBUG:teuthology.parallel:result is None 2024-09-15T01:31:32.411 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-mirror-17.2.7-1643.gd75f2382.el9.x86_64.rpm 8.7 MB/s | 2.9 MB 00:00 2024-09-15T01:31:32.411 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:32.412 INFO:teuthology.orchestra.run.smithi175.stdout:Total 8.7 MB/s | 2.9 MB 00:00 2024-09-15T01:31:32.412 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:31:32.421 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:31:32.421 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:31:32.500 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:31:32.501 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:31:32.929 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:31:32.995 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:33.020 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:33.020 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:31:33.021 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T01:31:33.021 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T01:31:33.021 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T01:31:33.021 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:34.153 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:34.153 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:34.153 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:31:34.153 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:34.154 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:34.154 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:34.414 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-nbd 2024-09-15T01:31:35.000 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:46 ago on Sun 15 Sep 2024 01:29:48 AM UTC. 2024-09-15T01:31:35.134 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-15T01:31:35.135 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:35.135 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-15T01:31:35.136 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:35.136 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-15T01:31:35.136 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 165 k 2024-09-15T01:31:35.136 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:35.136 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-15T01:31:35.136 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-15T01:31:35.136 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-15T01:31:35.136 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:35.136 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 165 k 2024-09-15T01:31:35.136 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 477 k 2024-09-15T01:31:35.136 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-15T01:31:35.356 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-nbd-17.2.7-1643.gd75f2382.el9.x86_64.rpm 754 kB/s | 165 kB 00:00 2024-09-15T01:31:35.356 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:35.357 INFO:teuthology.orchestra.run.smithi175.stdout:Total 748 kB/s | 165 kB 00:00 2024-09-15T01:31:35.357 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-15T01:31:35.364 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-15T01:31:35.364 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-15T01:31:35.428 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-15T01:31:35.429 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-15T01:31:35.589 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-15T01:31:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:36.363 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:36.363 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:36.363 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-15T01:31:36.363 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:36.363 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:36.363 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-15T01:31:36.608 DEBUG:teuthology.parallel:result is None 2024-09-15T01:31:36.609 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:31:36.764 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T01:31:36.789 INFO:teuthology.orchestra.run.smithi042.stdout:17.2.7-1643.gd75f2382.el9 2024-09-15T01:31:36.790 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643.gd75f2382.el9 2024-09-15T01:31:36.790 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643.gd75f2382 is installed. 2024-09-15T01:31:36.792 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:31:36.945 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T01:31:36.971 INFO:teuthology.orchestra.run.smithi115.stdout:17.2.7-1643.gd75f2382.el9 2024-09-15T01:31:36.971 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643.gd75f2382.el9 2024-09-15T01:31:36.971 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643.gd75f2382 is installed. 2024-09-15T01:31:36.973 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:31:37.116 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T01:31:37.141 INFO:teuthology.orchestra.run.smithi152.stdout:17.2.7-1643.gd75f2382.el9 2024-09-15T01:31:37.142 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643.gd75f2382.el9 2024-09-15T01:31:37.142 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643.gd75f2382 is installed. 2024-09-15T01:31:37.144 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:31:37.283 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T01:31:37.305 INFO:teuthology.orchestra.run.smithi175.stdout:17.2.7-1643.gd75f2382.el9 2024-09-15T01:31:37.305 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643.gd75f2382.el9 2024-09-15T01:31:37.305 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643.gd75f2382 is installed. 2024-09-15T01:31:37.307 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T01:31:37.307 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:37.307 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T01:31:37.341 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:31:37.341 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T01:31:37.372 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:31:37.372 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T01:31:37.404 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:37.404 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T01:31:37.430 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T01:31:37.431 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:37.431 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T01:31:37.459 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T01:31:37.527 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:31:37.527 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T01:31:37.557 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T01:31:37.626 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:31:37.627 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T01:31:37.656 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T01:31:37.725 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:37.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T01:31:37.751 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T01:31:37.817 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T01:31:37.817 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:37.818 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T01:31:37.847 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T01:31:37.918 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:31:37.918 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T01:31:37.943 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T01:31:38.008 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:31:38.008 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T01:31:38.038 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T01:31:38.107 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:38.108 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T01:31:38.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T01:31:38.201 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T01:31:38.201 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:38.201 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T01:31:38.233 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T01:31:38.301 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:31:38.302 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T01:31:38.327 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T01:31:38.392 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:31:38.392 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T01:31:38.422 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T01:31:38.492 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:38.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T01:31:38.519 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T01:31:38.585 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T01:31:38.661 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T01:31:38.661 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /var/log/ceph 2024-09-15T01:31:38.664 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /var/log/ceph 2024-09-15T01:31:38.666 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /var/log/ceph 2024-09-15T01:31:38.669 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /var/log/ceph 2024-09-15T01:31:38.695 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T01:31:38.696 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T01:31:38.735 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T01:31:38.737 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T01:31:38.739 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T01:31:38.766 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T01:31:38.766 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T01:31:38.804 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T01:31:38.806 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T01:31:38.809 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T01:31:38.847 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T01:31:38.847 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd sloppy crc': True}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T01:31:38.847 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045', 'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'description': 'rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7905045', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd sloppy crc': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'osd objectstore': 'filestore', 'osd sloppy crc': True}}, 'filestore': True, 'fs': 'xfs'}, 'install': {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 2126, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'targets': {'smithi042.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOip3M6JR7Vt1szwnIKkcuL/R+M/J9h+XcrLEzyY876hi7Wm4U3zZJ3PQon/Nz2Ebd24yhRaubFM54sx1pmH90E=', 'smithi115.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLg+RzEBGF5Qk4tevECApZMAdHoO9vuFaWkcyVtxY3AIZGMIqLtZFWVpyCt3ONGuLD/zo9ZGzf97c+iE9JsHgIE=', 'smithi152.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIiChYAueR05V+t/G16CAcbzoHUNAoQSNL8H59l+nDyh7hko8Oss5r/8fq3E08hkxR7T3VLEHD/uwg3cHdPCYjo=', 'smithi175.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVudi/xyXepdtqOz3uqgucJ9F2fNOMFiGS369WpVBG82r/veIETy39qNvFZw6Ef3Do25lyGbgdpcVS/RaHLYiQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd sloppy crc': True}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'cluster': 'ceph'}}, {'thrashosds': {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_00:54:39', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T01:31:38.848 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T01:31:38.875 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T01:31:38.877 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T01:31:38.880 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T01:31:38.895 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T01:31:38.932 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T01:31:38.934 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T01:31:38.937 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T01:31:38.962 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:38.963 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T01:31:39.015 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T01:31:39.015 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-09-15T01:31:39.072 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T01:31:39.073 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:39.073 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-15T01:31:39.073 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:39.073 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:39.073 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-15 01:31:28.522344826 +0000 2024-09-15T01:31:39.073 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-15 01:29:01.465191681 +0000 2024-09-15T01:31:39.073 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-15 01:29:01.465191681 +0000 2024-09-15T01:31:39.073 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-09-15 01:29:01.465191681 +0000 2024-09-15T01:31:39.073 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T01:31:39.138 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-15T01:31:39.138 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-15T01:31:39.138 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000105797 s, 4.8 MB/s 2024-09-15T01:31:39.140 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T01:31:39.198 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-09-15T01:31:39.255 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T01:31:39.255 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:39.255 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-15T01:31:39.255 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:39.255 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:39.255 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-15 01:31:28.522344826 +0000 2024-09-15T01:31:39.255 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-15 01:29:01.790188818 +0000 2024-09-15T01:31:39.255 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-15 01:29:01.790188818 +0000 2024-09-15T01:31:39.255 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-09-15 01:29:01.790188818 +0000 2024-09-15T01:31:39.256 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T01:31:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-15T01:31:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-15T01:31:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000111183 s, 4.6 MB/s 2024-09-15T01:31:39.322 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T01:31:39.379 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-09-15T01:31:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T01:31:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:39.436 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-15T01:31:39.436 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:39.436 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:39.436 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-15 01:31:28.522344826 +0000 2024-09-15T01:31:39.436 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-15 01:29:02.108186018 +0000 2024-09-15T01:31:39.436 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-15 01:29:02.108186018 +0000 2024-09-15T01:31:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-09-15 01:29:02.108186018 +0000 2024-09-15T01:31:39.436 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T01:31:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-15T01:31:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-15T01:31:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.00010167 s, 5.0 MB/s 2024-09-15T01:31:39.501 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T01:31:39.558 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-09-15T01:31:39.615 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T01:31:39.615 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-15T01:31:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-15 01:31:28.523344818 +0000 2024-09-15T01:31:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-15 01:29:02.438183111 +0000 2024-09-15T01:31:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-15 01:29:02.438183111 +0000 2024-09-15T01:31:39.615 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-09-15 01:29:02.438183111 +0000 2024-09-15T01:31:39.616 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T01:31:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-15T01:31:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-15T01:31:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000102683 s, 5.0 MB/s 2024-09-15T01:31:39.680 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T01:31:39.737 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:39.737 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:31:39.737 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T01:31:39.755 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T01:31:39.755 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-09-15T01:31:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T01:31:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:39.814 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-15T01:31:39.814 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:39.814 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:39.814 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-15 01:31:22.017880812 +0000 2024-09-15T01:31:39.814 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-15 01:28:58.778375631 +0000 2024-09-15T01:31:39.814 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-15 01:28:58.778375631 +0000 2024-09-15T01:31:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-15 01:28:58.778375631 +0000 2024-09-15T01:31:39.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T01:31:39.878 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-15T01:31:39.879 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-15T01:31:39.879 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000213574 s, 2.4 MB/s 2024-09-15T01:31:39.880 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T01:31:39.939 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-09-15T01:31:39.997 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T01:31:39.997 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:39.997 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-15T01:31:39.997 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:39.998 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:39.998 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-15 01:31:22.017880812 +0000 2024-09-15T01:31:39.998 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-15 01:28:59.111369421 +0000 2024-09-15T01:31:39.998 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-15 01:28:59.111369421 +0000 2024-09-15T01:31:39.998 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-15 01:28:59.111369421 +0000 2024-09-15T01:31:39.998 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T01:31:40.063 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-15T01:31:40.063 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-15T01:31:40.063 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000252629 s, 2.0 MB/s 2024-09-15T01:31:40.064 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T01:31:40.121 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-09-15T01:31:40.177 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T01:31:40.177 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:40.178 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-15T01:31:40.178 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:40.178 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:40.178 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-15 01:31:22.017880812 +0000 2024-09-15T01:31:40.178 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-15 01:28:59.432363434 +0000 2024-09-15T01:31:40.178 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-15 01:28:59.432363434 +0000 2024-09-15T01:31:40.178 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-15 01:28:59.432363434 +0000 2024-09-15T01:31:40.178 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T01:31:40.242 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-15T01:31:40.243 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-15T01:31:40.243 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000181768 s, 2.8 MB/s 2024-09-15T01:31:40.244 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T01:31:40.302 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-09-15T01:31:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T01:31:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:40.359 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-15T01:31:40.359 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:40.359 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:40.359 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-15 01:31:22.018880793 +0000 2024-09-15T01:31:40.359 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-15 01:28:59.742357653 +0000 2024-09-15T01:31:40.359 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-15 01:28:59.742357653 +0000 2024-09-15T01:31:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-15 01:28:59.742357653 +0000 2024-09-15T01:31:40.360 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T01:31:40.424 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-15T01:31:40.424 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-15T01:31:40.424 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000197446 s, 2.6 MB/s 2024-09-15T01:31:40.425 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T01:31:40.482 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:40.482 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:31:40.482 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T01:31:40.501 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T01:31:40.502 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2024-09-15T01:31:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T01:31:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:40.559 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-15T01:31:40.560 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:40.560 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:40.560 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-09-15 01:31:28.221228981 +0000 2024-09-15T01:31:40.560 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-09-15 01:29:00.271912717 +0000 2024-09-15T01:31:40.560 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-09-15 01:29:00.271912717 +0000 2024-09-15T01:31:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-09-15 01:29:00.271912717 +0000 2024-09-15T01:31:40.560 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T01:31:40.628 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-09-15T01:31:40.628 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-09-15T01:31:40.628 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000225621 s, 2.3 MB/s 2024-09-15T01:31:40.630 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T01:31:40.689 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2024-09-15T01:31:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T01:31:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:40.748 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-15T01:31:40.748 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:40.748 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:40.748 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-09-15 01:31:28.221228981 +0000 2024-09-15T01:31:40.748 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-09-15 01:29:00.590909033 +0000 2024-09-15T01:31:40.748 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-09-15 01:29:00.590909033 +0000 2024-09-15T01:31:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-09-15 01:29:00.590909033 +0000 2024-09-15T01:31:40.748 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T01:31:40.814 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-09-15T01:31:40.814 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-09-15T01:31:40.814 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000198572 s, 2.6 MB/s 2024-09-15T01:31:40.816 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T01:31:40.874 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2024-09-15T01:31:40.932 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T01:31:40.932 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:40.932 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-15T01:31:40.932 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:40.932 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:40.932 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-09-15 01:31:28.222228970 +0000 2024-09-15T01:31:40.932 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-09-15 01:29:00.902905431 +0000 2024-09-15T01:31:40.932 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-09-15 01:29:00.902905431 +0000 2024-09-15T01:31:40.932 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-09-15 01:29:00.902905431 +0000 2024-09-15T01:31:40.933 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T01:31:40.997 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-09-15T01:31:40.997 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-09-15T01:31:40.997 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000214081 s, 2.4 MB/s 2024-09-15T01:31:40.998 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T01:31:41.056 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2024-09-15T01:31:41.113 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T01:31:41.113 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:41.113 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-15T01:31:41.113 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:41.113 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:41.113 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-09-15 01:31:28.222228970 +0000 2024-09-15T01:31:41.113 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-09-15 01:29:01.234901598 +0000 2024-09-15T01:31:41.113 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-09-15 01:29:01.234901598 +0000 2024-09-15T01:31:41.113 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-09-15 01:29:01.234901598 +0000 2024-09-15T01:31:41.114 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T01:31:41.178 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-09-15T01:31:41.178 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-09-15T01:31:41.178 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000225012 s, 2.3 MB/s 2024-09-15T01:31:41.179 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T01:31:41.236 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:41.236 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:41.236 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T01:31:41.255 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T01:31:41.255 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-09-15T01:31:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T01:31:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:41.314 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-15T01:31:41.314 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:41.314 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:41.314 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-15 01:31:33.282940857 +0000 2024-09-15T01:31:41.314 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-15 01:29:00.806161874 +0000 2024-09-15T01:31:41.314 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-15 01:29:00.806161874 +0000 2024-09-15T01:31:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-15 01:29:00.806161874 +0000 2024-09-15T01:31:41.314 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T01:31:41.381 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-15T01:31:41.381 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-15T01:31:41.381 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.0003447 s, 1.5 MB/s 2024-09-15T01:31:41.382 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T01:31:41.441 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-09-15T01:31:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T01:31:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:41.499 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-15T01:31:41.499 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:41.499 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:41.499 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-15 01:31:33.282940857 +0000 2024-09-15T01:31:41.499 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-15 01:29:01.133156821 +0000 2024-09-15T01:31:41.499 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-15 01:29:01.133156821 +0000 2024-09-15T01:31:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-15 01:29:01.133156821 +0000 2024-09-15T01:31:41.499 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T01:31:41.564 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-15T01:31:41.564 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-15T01:31:41.564 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000222363 s, 2.3 MB/s 2024-09-15T01:31:41.565 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T01:31:41.622 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-09-15T01:31:41.679 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T01:31:41.679 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:41.679 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-15T01:31:41.679 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:41.679 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:41.679 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-15 01:31:33.283940842 +0000 2024-09-15T01:31:41.679 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-15 01:29:01.455151846 +0000 2024-09-15T01:31:41.680 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-15 01:29:01.455151846 +0000 2024-09-15T01:31:41.680 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-15 01:29:01.455151846 +0000 2024-09-15T01:31:41.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T01:31:41.744 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-15T01:31:41.744 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-15T01:31:41.744 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000221075 s, 2.3 MB/s 2024-09-15T01:31:41.745 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T01:31:41.801 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-09-15T01:31:41.857 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T01:31:41.857 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:41.857 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-15T01:31:41.857 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:41.858 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:41.858 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-15 01:31:33.283940842 +0000 2024-09-15T01:31:41.858 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-15 01:29:01.781146809 +0000 2024-09-15T01:31:41.858 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-15 01:29:01.781146809 +0000 2024-09-15T01:31:41.858 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-15 01:29:01.781146809 +0000 2024-09-15T01:31:41.858 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T01:31:41.921 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-15T01:31:41.921 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-15T01:31:41.921 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000192274 s, 2.7 MB/s 2024-09-15T01:31:41.922 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T01:31:41.979 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:41.979 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-09-15T01:31:41.979 INFO:tasks.ceph:Generating config... 2024-09-15T01:31:41.980 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-15T01:31:41.980 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-15T01:31:41.980 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T01:31:41.980 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T01:31:41.980 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.1 2024-09-15T01:31:41.980 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-15T01:31:41.980 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-15T01:31:41.980 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-09-15T01:31:41.980 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-15T01:31:41.981 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-15T01:31:41.981 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-15T01:31:41.981 INFO:tasks.ceph:[osd] osd objectstore = filestore 2024-09-15T01:31:41.981 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-15T01:31:41.981 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-15T01:31:41.981 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-09-15T01:31:41.981 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-09-15T01:31:41.981 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-15T01:31:41.981 INFO:tasks.ceph:[osd] osd sloppy crc = True 2024-09-15T01:31:41.981 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-15T01:31:41.981 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-15T01:31:41.981 INFO:tasks.ceph:[global] ms inject delay max = 1 2024-09-15T01:31:41.981 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2024-09-15T01:31:41.981 INFO:tasks.ceph:[global] ms inject delay type = osd 2024-09-15T01:31:41.982 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2024-09-15T01:31:41.982 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2024-09-15T01:31:41.982 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2024-09-15T01:31:41.982 INFO:tasks.ceph:[global] osd_object_clean_region_max_num_intervals = 1000 2024-09-15T01:31:41.982 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T01:31:41.982 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T01:31:41.982 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T01:31:41.982 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T01:31:41.982 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T01:31:41.982 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-15T01:31:41.982 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T01:31:41.982 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T01:31:42.039 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T01:31:42.042 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T01:31:42.085 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T01:31:42.153 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.42'), ('mon.b', '172.21.15.115'), ('mon.c', '172.21.15.152')] 2024-09-15T01:31:42.154 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.42,172.21.15.115,172.21.15.152', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd sloppy crc': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-15T01:31:42.155 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:42.155 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T01:31:42.211 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.42 --add b 172.21.15.115 --add c 172.21.15.152 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:42.294 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:42.295 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: generated fsid d4803586-ed44-4154-8ee1-6b95ec3e17e7 2024-09-15T01:31:42.295 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = octopus 2024-09-15T01:31:42.295 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 0 2024-09-15T01:31:42.295 INFO:teuthology.orchestra.run.smithi042.stdout:fsid d4803586-ed44-4154-8ee1-6b95ec3e17e7 2024-09-15T01:31:42.295 INFO:teuthology.orchestra.run.smithi042.stdout:last_changed 2024-09-15T01:31:42.293921+0000 2024-09-15T01:31:42.295 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-15T01:31:42.293921+0000 2024-09-15T01:31:42.295 INFO:teuthology.orchestra.run.smithi042.stdout:min_mon_release 15 (octopus) 2024-09-15T01:31:42.295 INFO:teuthology.orchestra.run.smithi042.stdout:election_strategy: 3 2024-09-15T01:31:42.295 INFO:teuthology.orchestra.run.smithi042.stdout:0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-09-15T01:31:42.295 INFO:teuthology.orchestra.run.smithi042.stdout:1: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-15T01:31:42.295 INFO:teuthology.orchestra.run.smithi042.stdout:2: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.c 2024-09-15T01:31:42.295 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T01:31:42.296 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T01:31:42.312 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d4803586-ed44-4154-8ee1-6b95ec3e17e7... 2024-09-15T01:31:42.313 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T01:31:42.355 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T01:31:42.357 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T01:31:42.360 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T01:31:42.398 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-09-15T01:31:42.398 INFO:teuthology.orchestra.run.smithi042.stdout: chdir = "" 2024-09-15T01:31:42.398 INFO:teuthology.orchestra.run.smithi042.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T01:31:42.398 INFO:teuthology.orchestra.run.smithi042.stdout: auth supported = cephx 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: filestore xattr use omap = true 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: mon clock drift allowed = 1.000 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: osd crush chooseleaf type = 0 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: auth debug = true 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on old message = true 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on bug = true 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T01:31:42.399 INFO:teuthology.orchestra.run.smithi042.stdout: mon pg warn max object skew = 0 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default size = 2 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow primary affinity = true 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow pg remap = true 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on legacy crush tunables = false 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on crush straw calc version zero = false 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on no sortbitwise = false 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on osd down out interval zero = false 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on too few osds = false 2024-09-15T01:31:42.400 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: mon_allow_pool_size_one = true 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: osd default data pool replay window = 5 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: mon allow pool delete = true 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: mon cluster log file level = debug 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: debug asserts on shutdown = true 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: mon health detail to clog = false 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: mon host = "172.21.15.42,172.21.15.115,172.21.15.152" 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi042.stdout: mon client directed command retry = 5 2024-09-15T01:31:42.402 INFO:teuthology.orchestra.run.smithi042.stdout: mon election default strategy = 3 2024-09-15T01:31:42.402 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject delay max = 1 2024-09-15T01:31:42.402 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject delay probability = 0.005 2024-09-15T01:31:42.402 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject delay type = osd 2024-09-15T01:31:42.402 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject internal delays = 0.002 2024-09-15T01:31:42.402 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject socket failures = 2500 2024-09-15T01:31:42.402 INFO:teuthology.orchestra.run.smithi042.stdout: osd_async_recovery_min_cost = 1 2024-09-15T01:31:42.402 INFO:teuthology.orchestra.run.smithi042.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-15T01:31:42.402 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = d4803586-ed44-4154-8ee1-6b95ec3e17e7 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout:[osd] 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: osd journal size = 100 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub load threshold = 5.0 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub max interval = 120 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: osd mclock profile = high_recovery_ops 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: osd recover clone overlap = true 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: osd recovery max chunk = 1048576 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug shutdown = true 2024-09-15T01:31:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug op order = true 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug verify stray on activate = true 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: osd open classes on start = true 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug pg log writeout = true 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: osd deep scrub update digest min age = 30 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: osd map max advance = 10 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: journal zero on create = true 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: filestore ondisk finisher threads = 3 2024-09-15T01:31:42.404 INFO:teuthology.orchestra.run.smithi042.stdout: filestore apply finisher threads = 3 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: bdev debug aio = true 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug misdirected ops = true 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: debug monc = 20 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore zero block detection = True 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: debug osd = 20 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug reject backfill probability = 0.1 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug verify cached snaps = True 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug verify missing on start = True 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: osd max backfills = 3 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: osd max markdown count = 1000 2024-09-15T01:31:42.405 INFO:teuthology.orchestra.run.smithi042.stdout: osd mclock override recovery settings = True 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout: osd objectstore = filestore 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout: osd op queue = debug_random 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout: osd op queue cut off = debug_random 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub min interval = 60 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout: osd shutdown pgref assert = True 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout: osd sloppy crc = True 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr] 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout: debug mgr = 20 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-09-15T01:31:42.406 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:31:42.407 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:31:42.407 INFO:teuthology.orchestra.run.smithi042.stdout: mgr/telemetry/nag = false 2024-09-15T01:31:42.407 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.407 INFO:teuthology.orchestra.run.smithi042.stdout:[mon] 2024-09-15T01:31:42.407 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-09-15T01:31:42.407 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-09-15T01:31:42.407 INFO:teuthology.orchestra.run.smithi042.stdout: debug paxos = 20 2024-09-15T01:31:42.407 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-09-15T01:31:42.407 INFO:teuthology.orchestra.run.smithi042.stdout: mon data avail warn = 5 2024-09-15T01:31:42.407 INFO:teuthology.orchestra.run.smithi042.stdout: mon mgr mkfs grace = 240 2024-09-15T01:31:42.407 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:31:42.407 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd reporter subtree level = osd 2024-09-15T01:31:42.408 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd prime pg temp = true 2024-09-15T01:31:42.408 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:31:42.408 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.408 INFO:teuthology.orchestra.run.smithi042.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T01:31:42.408 INFO:teuthology.orchestra.run.smithi042.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T01:31:42.408 INFO:teuthology.orchestra.run.smithi042.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T01:31:42.408 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.408 INFO:teuthology.orchestra.run.smithi042.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T01:31:42.408 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T01:31:42.408 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout: # 1m isn't quite enough 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout: mon_down_mkfs_grace = 2m 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_filestore_osds = false 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout: mon scrub interval = 300 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout:[client] 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout: rgw cache enabled = true 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable ops log = true 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable usage log = true 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T01:31:42.409 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.a] 2024-09-15T01:31:42.410 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.b] 2024-09-15T01:31:42.410 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.c] 2024-09-15T01:31:42.411 INFO:teuthology.orchestra.run.smithi152.stdout:[global] 2024-09-15T01:31:42.411 INFO:teuthology.orchestra.run.smithi152.stdout: chdir = "" 2024-09-15T01:31:42.411 INFO:teuthology.orchestra.run.smithi152.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T01:31:42.411 INFO:teuthology.orchestra.run.smithi152.stdout: auth supported = cephx 2024-09-15T01:31:42.411 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.411 INFO:teuthology.orchestra.run.smithi152.stdout: filestore xattr use omap = true 2024-09-15T01:31:42.411 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.411 INFO:teuthology.orchestra.run.smithi152.stdout: mon clock drift allowed = 1.000 2024-09-15T01:31:42.411 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.411 INFO:teuthology.orchestra.run.smithi152.stdout: osd crush chooseleaf type = 0 2024-09-15T01:31:42.411 INFO:teuthology.orchestra.run.smithi152.stdout: auth debug = true 2024-09-15T01:31:42.411 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.412 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on old message = true 2024-09-15T01:31:42.412 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on bug = true 2024-09-15T01:31:42.412 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.412 INFO:teuthology.orchestra.run.smithi152.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T01:31:42.412 INFO:teuthology.orchestra.run.smithi152.stdout: mon pg warn max object skew = 0 2024-09-15T01:31:42.412 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.412 INFO:teuthology.orchestra.run.smithi152.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T01:31:42.412 INFO:teuthology.orchestra.run.smithi152.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T01:31:42.412 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.412 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default size = 2 2024-09-15T01:31:42.412 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.412 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow primary affinity = true 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow pg remap = true 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on legacy crush tunables = false 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on crush straw calc version zero = false 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on no sortbitwise = false 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on osd down out interval zero = false 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on too few osds = false 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: mon_allow_pool_size_one = true 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.413 INFO:teuthology.orchestra.run.smithi152.stdout: osd default data pool replay window = 5 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: mon allow pool delete = true 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: mon cluster log file level = debug 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: debug asserts on shutdown = true 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: mon health detail to clog = false 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: mon host = "172.21.15.42,172.21.15.115,172.21.15.152" 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: mon client directed command retry = 5 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: mon election default strategy = 3 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: ms inject delay max = 1 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: ms inject delay probability = 0.005 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: ms inject delay type = osd 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: ms inject internal delays = 0.002 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: ms inject socket failures = 2500 2024-09-15T01:31:42.414 INFO:teuthology.orchestra.run.smithi152.stdout: osd_async_recovery_min_cost = 1 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout: fsid = d4803586-ed44-4154-8ee1-6b95ec3e17e7 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout:[osd] 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout: osd journal size = 100 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub load threshold = 5.0 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub max interval = 120 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout: osd mclock profile = high_recovery_ops 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout: osd recover clone overlap = true 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout: osd recovery max chunk = 1048576 2024-09-15T01:31:42.415 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug shutdown = true 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug op order = true 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify stray on activate = true 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: osd open classes on start = true 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug pg log writeout = true 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: osd deep scrub update digest min age = 30 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: osd map max advance = 10 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: journal zero on create = true 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: filestore ondisk finisher threads = 3 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: filestore apply finisher threads = 3 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: bdev debug aio = true 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug misdirected ops = true 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: debug monc = 20 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore zero block detection = True 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: debug osd = 20 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug reject backfill probability = 0.1 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify cached snaps = True 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify missing on start = True 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: osd max backfills = 3 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: osd max markdown count = 1000 2024-09-15T01:31:42.417 INFO:teuthology.orchestra.run.smithi152.stdout: osd mclock override recovery settings = True 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout: osd objectstore = filestore 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout: osd op queue = debug_random 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout: osd op queue cut off = debug_random 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub min interval = 60 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout: osd shutdown pgref assert = True 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout: osd sloppy crc = True 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr] 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout: debug mgr = 20 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2024-09-15T01:31:42.418 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: mgr/telemetry/nag = false 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout:[mon] 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: debug paxos = 20 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: mon data avail warn = 5 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: mon mgr mkfs grace = 240 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd reporter subtree level = osd 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd prime pg temp = true 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:31:42.419 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: # 1m isn't quite enough 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: mon_down_mkfs_grace = 2m 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_filestore_osds = false 2024-09-15T01:31:42.420 INFO:teuthology.orchestra.run.smithi152.stdout: mon scrub interval = 300 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi152.stdout:[client] 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi152.stdout: rgw cache enabled = true 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable ops log = true 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable usage log = true 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi152.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi152.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.a] 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.b] 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.c] 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2024-09-15T01:31:42.421 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2024-09-15T01:31:42.422 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T01:31:42.423 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.42,172.21.15.115,172.21.15.152" 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: mon client directed command retry = 5 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: mon election default strategy = 3 2024-09-15T01:31:42.424 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject delay max = 1 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject delay probability = 0.005 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject delay type = osd 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject internal delays = 0.002 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject socket failures = 2500 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: osd_async_recovery_min_cost = 1 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = d4803586-ed44-4154-8ee1-6b95ec3e17e7 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 120 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2024-09-15T01:31:42.425 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.426 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: debug monc = 20 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore zero block detection = True 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug reject backfill probability = 0.1 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify cached snaps = True 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify missing on start = True 2024-09-15T01:31:42.427 INFO:teuthology.orchestra.run.smithi175.stdout: osd max backfills = 3 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: osd max markdown count = 1000 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock override recovery settings = True 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = filestore 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: osd op queue = debug_random 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: osd op queue cut off = debug_random 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub min interval = 60 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: osd shutdown pgref assert = True 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: osd sloppy crc = True 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-09-15T01:31:42.428 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2024-09-15T01:31:42.429 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: mon scrub interval = 300 2024-09-15T01:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi115.stdout: chdir = "" 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi115.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi115.stdout: auth supported = cephx 2024-09-15T01:31:42.431 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: filestore xattr use omap = true 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: mon clock drift allowed = 1.000 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: osd crush chooseleaf type = 0 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: auth debug = true 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on old message = true 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on bug = true 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: mon pg warn max object skew = 0 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.432 INFO:teuthology.orchestra.run.smithi115.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default size = 2 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow primary affinity = true 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow pg remap = true 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on legacy crush tunables = false 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on crush straw calc version zero = false 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on no sortbitwise = false 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on osd down out interval zero = false 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on too few osds = false 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T01:31:42.433 INFO:teuthology.orchestra.run.smithi115.stdout: mon_allow_pool_size_one = true 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: osd default data pool replay window = 5 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: mon allow pool delete = true 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: mon cluster log file level = debug 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: debug asserts on shutdown = true 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: mon health detail to clog = false 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: mon host = "172.21.15.42,172.21.15.115,172.21.15.152" 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: mon client directed command retry = 5 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: mon election default strategy = 3 2024-09-15T01:31:42.434 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject delay max = 1 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject delay probability = 0.005 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject delay type = osd 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject internal delays = 0.002 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject socket failures = 2500 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: osd_async_recovery_min_cost = 1 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = d4803586-ed44-4154-8ee1-6b95ec3e17e7 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout:[osd] 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: osd journal size = 100 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub load threshold = 5.0 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub max interval = 120 2024-09-15T01:31:42.435 INFO:teuthology.orchestra.run.smithi115.stdout: osd mclock profile = high_recovery_ops 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: osd recover clone overlap = true 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: osd recovery max chunk = 1048576 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug shutdown = true 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug op order = true 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify stray on activate = true 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: osd open classes on start = true 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug pg log writeout = true 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: osd deep scrub update digest min age = 30 2024-09-15T01:31:42.436 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: osd map max advance = 10 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: journal zero on create = true 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: filestore ondisk finisher threads = 3 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: filestore apply finisher threads = 3 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: bdev debug aio = true 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug misdirected ops = true 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: debug monc = 20 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore zero block detection = True 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: debug osd = 20 2024-09-15T01:31:42.437 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug reject backfill probability = 0.1 2024-09-15T01:31:42.438 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify cached snaps = True 2024-09-15T01:31:42.438 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify missing on start = True 2024-09-15T01:31:42.438 INFO:teuthology.orchestra.run.smithi115.stdout: osd max backfills = 3 2024-09-15T01:31:42.438 INFO:teuthology.orchestra.run.smithi115.stdout: osd max markdown count = 1000 2024-09-15T01:31:42.438 INFO:teuthology.orchestra.run.smithi115.stdout: osd mclock override recovery settings = True 2024-09-15T01:31:42.438 INFO:teuthology.orchestra.run.smithi115.stdout: osd objectstore = filestore 2024-09-15T01:31:42.438 INFO:teuthology.orchestra.run.smithi115.stdout: osd op queue = debug_random 2024-09-15T01:31:42.438 INFO:teuthology.orchestra.run.smithi115.stdout: osd op queue cut off = debug_random 2024-09-15T01:31:42.438 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub min interval = 60 2024-09-15T01:31:42.438 INFO:teuthology.orchestra.run.smithi115.stdout: osd shutdown pgref assert = True 2024-09-15T01:31:42.438 INFO:teuthology.orchestra.run.smithi115.stdout: osd sloppy crc = True 2024-09-15T01:31:42.439 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.439 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr] 2024-09-15T01:31:42.439 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-15T01:31:42.439 INFO:teuthology.orchestra.run.smithi115.stdout: debug mgr = 20 2024-09-15T01:31:42.439 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-09-15T01:31:42.439 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-09-15T01:31:42.439 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:31:42.439 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:31:42.439 INFO:teuthology.orchestra.run.smithi115.stdout: mgr/telemetry/nag = false 2024-09-15T01:31:42.439 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.439 INFO:teuthology.orchestra.run.smithi115.stdout:[mon] 2024-09-15T01:31:42.439 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-15T01:31:42.440 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-09-15T01:31:42.440 INFO:teuthology.orchestra.run.smithi115.stdout: debug paxos = 20 2024-09-15T01:31:42.440 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-09-15T01:31:42.440 INFO:teuthology.orchestra.run.smithi115.stdout: mon data avail warn = 5 2024-09-15T01:31:42.440 INFO:teuthology.orchestra.run.smithi115.stdout: mon mgr mkfs grace = 240 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd reporter subtree level = osd 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd prime pg temp = true 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T01:31:42.441 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout: # 1m isn't quite enough 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout: mon_down_mkfs_grace = 2m 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_filestore_osds = false 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout: mon scrub interval = 300 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout:[client] 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout: rgw cache enabled = true 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable ops log = true 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable usage log = true 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.a] 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.b] 2024-09-15T01:31:42.442 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.c] 2024-09-15T01:31:42.445 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T01:31:42.445 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T01:31:42.535 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T01:31:42.535 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:42.535 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T01:31:42.551 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:42.551 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T01:31:42.607 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T01:31:42.607 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:42.607 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T01:31:42.608 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T01:31:42.683 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:42.683 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:42.739 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi115.front.sepia.ceph.com 2024-09-15T01:31:42.740 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:31:42.740 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T01:31:42.740 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T01:31:42.779 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:31:42.779 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:42.836 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi152.front.sepia.ceph.com 2024-09-15T01:31:42.837 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:31:42.837 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T01:31:42.837 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T01:31:42.879 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:31:42.879 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:42.937 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi175.front.sepia.ceph.com 2024-09-15T01:31:42.938 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:42.938 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T01:31:42.938 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T01:31:42.976 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:42.976 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:43.032 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T01:31:43.032 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T01:31:43.032 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T01:31:43.082 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T01:31:43.084 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T01:31:43.165 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T01:31:43.168 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T01:31:43.168 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T01:31:43.168 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T01:31:43.205 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T01:31:43.217 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T01:31:43.217 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-09-15T01:31:43.217 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T01:31:43.243 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:43.244 INFO:tasks.ceph:role: osd.0 2024-09-15T01:31:43.244 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T01:31:43.244 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T01:31:43.309 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:43.309 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:43.309 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:43.309 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:43.341 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-09-15T01:31:43.343 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-15T01:31:43.343 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T01:31:43.413 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T01:31:43.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-15T01:31:43.547 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:43.547 INFO:tasks.ceph:role: osd.4 2024-09-15T01:31:43.547 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T01:31:43.547 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T01:31:43.612 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:43.612 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:43.613 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:43.613 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:43.613 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:43.613 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:43.613 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:43.645 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-09-15T01:31:43.647 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-15T01:31:43.647 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-15T01:31:43.717 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-15T01:31:43.787 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-09-15T01:31:43.853 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:43.853 INFO:tasks.ceph:role: osd.8 2024-09-15T01:31:43.853 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T01:31:43.853 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T01:31:43.920 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:43.920 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:43.920 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:43.920 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:43.920 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:43.952 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-09-15T01:31:43.954 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-15T01:31:43.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2024-09-15T01:31:44.025 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2024-09-15T01:31:44.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2024-09-15T01:31:44.162 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:44.162 INFO:tasks.ceph:role: osd.12 2024-09-15T01:31:44.162 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T01:31:44.162 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T01:31:44.227 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:44.227 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:44.227 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:44.227 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:44.227 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:44.227 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:44.227 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:44.227 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:44.227 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:44.227 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:44.259 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-09-15T01:31:44.261 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-15T01:31:44.261 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2024-09-15T01:31:44.332 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2024-09-15T01:31:44.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:44.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.503+0000 7f3d2a489440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T01:31:44.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.503+0000 7f3d2a489440 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T01:31:44.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.507+0000 7f3d2a489440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:44.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.507+0000 7f3d2a489440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:44.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.565+0000 7f3d2a489440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:44.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.565+0000 7f3d2a489440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:44.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.565+0000 7f3d2a489440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:44.573 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T01:31:44.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:44.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.678+0000 7f6d44323440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-15T01:31:44.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.678+0000 7f6d44323440 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-15T01:31:44.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.683+0000 7f6d44323440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:44.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.683+0000 7f6d44323440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.739+0000 7f6d44323440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.739+0000 7f6d44323440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.739+0000 7f6d44323440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:44.748 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-15T01:31:44.814 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:44.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.895+0000 7fa3f0c8b440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-09-15T01:31:44.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.896+0000 7fa3f0c8b440 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-09-15T01:31:44.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.900+0000 7fa3f0c8b440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:44.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.900+0000 7fa3f0c8b440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.958+0000 7fa3f0c8b440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.958+0000 7fa3f0c8b440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:44.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:44.958+0000 7fa3f0c8b440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:44.966 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-09-15T01:31:45.033 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:45.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:45.114+0000 7f674963c440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2024-09-15T01:31:45.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:45.114+0000 7f674963c440 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2024-09-15T01:31:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:45.118+0000 7f674963c440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:45.118+0000 7f674963c440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:45.174+0000 7f674963c440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:45.174+0000 7f674963c440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:31:45.174+0000 7f674963c440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:45.182 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2024-09-15T01:31:45.248 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T01:31:45.278 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:45.278 INFO:tasks.ceph:role: osd.1 2024-09-15T01:31:45.278 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-15T01:31:45.278 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T01:31:45.355 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:45.355 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:45.356 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:45.356 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:45.356 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:45.844 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-15T01:31:45.846 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-15T01:31:45.846 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-09-15T01:31:45.983 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T01:31:46.016 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-15T01:31:46.082 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:46.082 INFO:tasks.ceph:role: osd.5 2024-09-15T01:31:46.082 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-15T01:31:46.082 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T01:31:46.155 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:46.156 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:46.156 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:46.156 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:46.156 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:46.617 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-15T01:31:46.620 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-15T01:31:46.620 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-15T01:31:46.712 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-15T01:31:46.742 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-09-15T01:31:46.807 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:46.808 INFO:tasks.ceph:role: osd.9 2024-09-15T01:31:46.808 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-15T01:31:46.808 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T01:31:46.880 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:46.881 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:46.881 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:46.881 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:46.881 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:46.881 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:46.881 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:47.375 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-15T01:31:47.377 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-15T01:31:47.377 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2024-09-15T01:31:47.518 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2024-09-15T01:31:47.551 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2024-09-15T01:31:47.618 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:47.619 INFO:tasks.ceph:role: osd.13 2024-09-15T01:31:47.619 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-15T01:31:47.619 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T01:31:47.693 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:47.693 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:47.693 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:47.693 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:47.693 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:47.693 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:47.693 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:47.693 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:47.694 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:47.694 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-15T01:31:48.219 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-15T01:31:48.219 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2024-09-15T01:31:48.375 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2024-09-15T01:31:48.409 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:48.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:48.505+0000 7f265e923440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T01:31:48.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:48.506+0000 7f265e923440 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T01:31:48.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:48.515+0000 7f265e923440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:48.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:48.515+0000 7f265e923440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:48.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:48.735+0000 7f265e923440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:48.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:48.735+0000 7f265e923440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:48.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:48.735+0000 7f265e923440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:48.763 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T01:31:48.789 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:48.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:48.869+0000 7f506621f440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-15T01:31:48.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:48.869+0000 7f506621f440 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-15T01:31:48.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:48.874+0000 7f506621f440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:48.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:48.874+0000 7f506621f440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:49.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.022+0000 7f506621f440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:49.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.022+0000 7f506621f440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:49.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.022+0000 7f506621f440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:49.031 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-15T01:31:49.096 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:49.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.175+0000 7f49ea92d440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-09-15T01:31:49.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.175+0000 7f49ea92d440 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-09-15T01:31:49.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.179+0000 7f49ea92d440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:49.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.179+0000 7f49ea92d440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:49.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.298+0000 7f49ea92d440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:49.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.298+0000 7f49ea92d440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:49.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.298+0000 7f49ea92d440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:49.308 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-09-15T01:31:49.373 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:49.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.452+0000 7f2a7ce3b440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2024-09-15T01:31:49.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.452+0000 7f2a7ce3b440 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2024-09-15T01:31:49.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.456+0000 7f2a7ce3b440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:49.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.456+0000 7f2a7ce3b440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:49.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.586+0000 7f2a7ce3b440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:49.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.586+0000 7f2a7ce3b440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:49.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-15T01:31:49.586+0000 7f2a7ce3b440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:49.595 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2024-09-15T01:31:49.660 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T01:31:49.691 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:49.691 INFO:tasks.ceph:role: osd.2 2024-09-15T01:31:49.691 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com 2024-09-15T01:31:49.691 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T01:31:49.766 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:49.767 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:49.767 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:49.767 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:49.767 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:50.295 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-09-15T01:31:50.297 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-09-15T01:31:50.297 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-09-15T01:31:50.453 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T01:31:50.489 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-15T01:31:50.558 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:50.558 INFO:tasks.ceph:role: osd.6 2024-09-15T01:31:50.558 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com 2024-09-15T01:31:50.558 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T01:31:50.632 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:50.632 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:50.632 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:50.633 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:50.633 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:50.633 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:50.633 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:50.633 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:50.633 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:50.633 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:51.079 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-09-15T01:31:51.082 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-09-15T01:31:51.082 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2024-09-15T01:31:51.176 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-15T01:31:51.208 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2024-09-15T01:31:51.272 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:51.272 INFO:tasks.ceph:role: osd.10 2024-09-15T01:31:51.272 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com 2024-09-15T01:31:51.272 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T01:31:51.345 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:51.345 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:51.345 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:51.345 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:51.345 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:51.835 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-09-15T01:31:51.837 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-09-15T01:31:51.837 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2024-09-15T01:31:51.978 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2024-09-15T01:31:52.008 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2024-09-15T01:31:52.073 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:52.073 INFO:tasks.ceph:role: osd.14 2024-09-15T01:31:52.073 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com 2024-09-15T01:31:52.073 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T01:31:52.145 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:52.146 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:52.146 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:52.146 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:52.146 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:52.146 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:52.146 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:52.146 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:52.146 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:52.146 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:52.622 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-09-15T01:31:52.625 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-09-15T01:31:52.625 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2024-09-15T01:31:52.752 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2024-09-15T01:31:52.784 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:52.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:52.880+0000 7ff549c4d440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T01:31:52.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:52.881+0000 7ff549c4d440 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T01:31:52.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:52.885+0000 7ff549c4d440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:52.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:52.885+0000 7ff549c4d440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:53.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.034+0000 7ff549c4d440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:53.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.034+0000 7ff549c4d440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:53.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.034+0000 7ff549c4d440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:53.045 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T01:31:53.069 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:53.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.148+0000 7fb6d951c440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-15T01:31:53.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.148+0000 7fb6d951c440 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-15T01:31:53.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.152+0000 7fb6d951c440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:53.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.153+0000 7fb6d951c440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:53.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.282+0000 7fb6d951c440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:53.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.283+0000 7fb6d951c440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:53.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.283+0000 7fb6d951c440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:53.294 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-15T01:31:53.359 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:53.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.437+0000 7f442564f440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2024-09-15T01:31:53.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.437+0000 7f442564f440 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2024-09-15T01:31:53.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.443+0000 7f442564f440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:53.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.443+0000 7f442564f440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:53.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.577+0000 7f442564f440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:53.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.578+0000 7f442564f440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:53.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.578+0000 7f442564f440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:53.590 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2024-09-15T01:31:53.655 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:53.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.733+0000 7f5b18615440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2024-09-15T01:31:53.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.734+0000 7f5b18615440 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.739+0000 7f5b18615440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.739+0000 7f5b18615440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:53.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.873+0000 7f5b18615440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:53.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.874+0000 7f5b18615440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:53.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-15T01:31:53.874+0000 7f5b18615440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:53.883 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2024-09-15T01:31:53.949 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T01:31:53.978 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:53.978 INFO:tasks.ceph:role: osd.3 2024-09-15T01:31:53.978 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-15T01:31:53.978 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T01:31:54.055 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:54.056 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:54.056 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:54.056 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:54.056 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:54.056 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:54.056 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:54.056 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:54.056 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:54.056 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-09-15T01:31:54.553 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-09-15T01:31:54.554 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-15T01:31:54.684 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T01:31:54.718 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-15T01:31:54.784 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:54.784 INFO:tasks.ceph:role: osd.7 2024-09-15T01:31:54.784 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-15T01:31:54.785 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T01:31:54.858 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:54.859 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:54.859 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:54.859 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:54.859 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:54.859 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:54.859 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:54.859 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:54.859 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:54.859 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:55.317 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-09-15T01:31:55.319 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-09-15T01:31:55.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2024-09-15T01:31:55.408 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-15T01:31:55.436 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2024-09-15T01:31:55.501 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:55.501 INFO:tasks.ceph:role: osd.11 2024-09-15T01:31:55.501 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-15T01:31:55.501 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T01:31:55.574 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:55.575 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:55.576 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:55.576 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:55.576 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:56.065 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-09-15T01:31:56.067 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-09-15T01:31:56.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2024-09-15T01:31:56.202 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2024-09-15T01:31:56.233 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2024-09-15T01:31:56.297 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-15T01:31:56.298 INFO:tasks.ceph:role: osd.15 2024-09-15T01:31:56.298 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-15T01:31:56.298 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T01:31:56.369 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:56.378 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:56.378 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:56.378 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:56.379 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:56.900 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-09-15T01:31:56.902 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-09-15T01:31:56.902 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2024-09-15T01:31:57.052 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2024-09-15T01:31:57.085 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:57.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.187+0000 7f713eb21440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T01:31:57.189 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.188+0000 7f713eb21440 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T01:31:57.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.192+0000 7f713eb21440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:57.194 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.192+0000 7f713eb21440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:57.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.395+0000 7f713eb21440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:57.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.395+0000 7f713eb21440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:57.397 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.395+0000 7f713eb21440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:57.411 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T01:31:57.435 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:57.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.513+0000 7fa117b21440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-15T01:31:57.515 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.513+0000 7fa117b21440 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-15T01:31:57.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.518+0000 7fa117b21440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:57.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.518+0000 7fa117b21440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:57.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.662+0000 7fa117b21440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:57.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.662+0000 7fa117b21440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:57.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.662+0000 7fa117b21440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:57.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-15T01:31:57.737 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:57.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.816+0000 7f2b1fe09440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2024-09-15T01:31:57.818 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.816+0000 7f2b1fe09440 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2024-09-15T01:31:57.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.820+0000 7f2b1fe09440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:57.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.820+0000 7f2b1fe09440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:57.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.939+0000 7f2b1fe09440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:57.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.939+0000 7f2b1fe09440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:57.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:57.939+0000 7f2b1fe09440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:57.951 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2024-09-15T01:31:58.016 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:58.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:58.095+0000 7f4948d23440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2024-09-15T01:31:58.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:58.095+0000 7f4948d23440 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2024-09-15T01:31:58.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:58.102+0000 7f4948d23440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:58.104 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:58.102+0000 7f4948d23440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:58.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:58.228+0000 7f4948d23440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:58.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:58.230+0000 7f4948d23440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:58.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-15T01:31:58.230+0000 7f4948d23440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:58.241 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2024-09-15T01:31:58.305 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T01:31:58.305 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:58.305 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-15T01:31:58.337 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:58.337 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T01:31:58.409 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:58.409 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-15T01:31:58.478 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:58.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-09-15T01:31:58.548 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:31:58.548 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2024-09-15T01:31:58.616 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:31:58.616 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T01:31:58.646 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:31:58.647 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-15T01:31:58.716 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:31:58.716 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-09-15T01:31:58.783 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:31:58.783 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2024-09-15T01:31:58.850 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:31:58.850 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T01:31:58.880 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:31:58.880 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-15T01:31:58.950 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:31:58.950 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2024-09-15T01:31:59.018 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:31:59.018 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2024-09-15T01:31:59.085 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:59.085 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T01:31:59.113 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:59.113 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T01:31:59.180 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:59.180 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-15T01:31:59.247 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:59.247 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2024-09-15T01:31:59.314 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:59.314 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2024-09-15T01:31:59.380 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:31:59.381 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T01:31:59.437 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T01:31:59.437 DEBUG:teuthology.orchestra.run.smithi042:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T01:31:59.440 DEBUG:teuthology.orchestra.run.smithi115:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T01:31:59.442 DEBUG:teuthology.orchestra.run.smithi152:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T01:31:59.467 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.y] 2024-09-15T01:31:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQD/OOZmOhXdBBAA5xLh1V9I8jMCTs5dg2Okqw== 2024-09-15T01:31:59.467 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.0] 2024-09-15T01:31:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAAOeZmanEPHhAAxtaj43OTEU5YFd9531kdeg== 2024-09-15T01:31:59.467 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.4] 2024-09-15T01:31:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAAOeZmfDOAKBAAYrV4ihrCbsVgL79ZLuTpbg== 2024-09-15T01:31:59.467 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.8] 2024-09-15T01:31:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAAOeZmitpxNRAAUIZZlxi+pkN2Iy8+k+X8fQ== 2024-09-15T01:31:59.467 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.12] 2024-09-15T01:31:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQABOeZmqRnaBhAAFG/5Gp7PAc7jtN31aPXvdg== 2024-09-15T01:31:59.467 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.1] 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAEOeZmLXIwHhAAvK93eEU04mG8m6v86JJ81w== 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.5] 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAEOeZmCJrbMxAAvHVAmDW6nVOcrqI337msXw== 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.9] 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAFOeZmVXx6ChAAOJjfSnplmOpdURgEXtfW+g== 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.13] 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAFOeZmL7v8GhAA6bBw8aYy/6jjnln3h9yr/Q== 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.2] 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAIOeZmIDeaNBAAs2UHTFr0xO65mu0He+5I7Q== 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.6] 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAJOeZmi070CBAApu6D1pPV946BlmaCKLieag== 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.10] 2024-09-15T01:31:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAJOeZmHW0yGhAAyVcSN6fsP+38isadzo2VzA== 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.14] 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAJOeZmzF/YKxAAqGM9VUnEXwHVtFy4Fq6gEQ== 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.x] 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQD/OOZmUV/VCRAAArjdChxqFa363iy+INXmmQ== 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.3] 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQANOeZmpW0/CxAAoPtyi9oc8T0V2K53ErhJtA== 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.7] 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQANOeZmyjGqHhAAM2eUIgH5Xd4HxjODEKKC7Q== 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.11] 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQANOeZmtdC0MBAAOD+I7kAS9mn+xyntNOZI5g== 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.15] 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAOOeZmnPu/BRAA85tgq1GT8qvxZx5Pw+kGxQ== 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-09-15T01:31:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQD/OOZmeTUrDBAA+wrChYXhqHdtZoGm/2q/ug== 2024-09-15T01:31:59.470 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.y] 2024-09-15T01:31:59.470 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQD/OOZmOhXdBBAA5xLh1V9I8jMCTs5dg2Okqw== 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.0] 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQAAOeZmanEPHhAAxtaj43OTEU5YFd9531kdeg== 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.4] 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQAAOeZmfDOAKBAAYrV4ihrCbsVgL79ZLuTpbg== 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.8] 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQAAOeZmitpxNRAAUIZZlxi+pkN2Iy8+k+X8fQ== 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.12] 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQABOeZmqRnaBhAAFG/5Gp7PAc7jtN31aPXvdg== 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.1] 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQAEOeZmLXIwHhAAvK93eEU04mG8m6v86JJ81w== 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.5] 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQAEOeZmCJrbMxAAvHVAmDW6nVOcrqI337msXw== 2024-09-15T01:31:59.471 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.9] 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQAFOeZmVXx6ChAAOJjfSnplmOpdURgEXtfW+g== 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.13] 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQAFOeZmL7v8GhAA6bBw8aYy/6jjnln3h9yr/Q== 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.2] 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQAIOeZmIDeaNBAAs2UHTFr0xO65mu0He+5I7Q== 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.6] 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQAJOeZmi070CBAApu6D1pPV946BlmaCKLieag== 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.10] 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQAJOeZmHW0yGhAAyVcSN6fsP+38isadzo2VzA== 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.14] 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQAJOeZmzF/YKxAAqGM9VUnEXwHVtFy4Fq6gEQ== 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.x] 2024-09-15T01:31:59.472 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQD/OOZmUV/VCRAAArjdChxqFa363iy+INXmmQ== 2024-09-15T01:31:59.473 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.3] 2024-09-15T01:31:59.473 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQANOeZmpW0/CxAAoPtyi9oc8T0V2K53ErhJtA== 2024-09-15T01:31:59.473 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.7] 2024-09-15T01:31:59.473 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQANOeZmyjGqHhAAM2eUIgH5Xd4HxjODEKKC7Q== 2024-09-15T01:31:59.473 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.11] 2024-09-15T01:31:59.473 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQANOeZmtdC0MBAAOD+I7kAS9mn+xyntNOZI5g== 2024-09-15T01:31:59.473 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.15] 2024-09-15T01:31:59.473 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQAOOeZmnPu/BRAA85tgq1GT8qvxZx5Pw+kGxQ== 2024-09-15T01:31:59.473 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-09-15T01:31:59.473 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQD/OOZmeTUrDBAA+wrChYXhqHdtZoGm/2q/ug== 2024-09-15T01:31:59.474 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.y] 2024-09-15T01:31:59.474 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQD/OOZmOhXdBBAA5xLh1V9I8jMCTs5dg2Okqw== 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.0] 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAAOeZmanEPHhAAxtaj43OTEU5YFd9531kdeg== 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.4] 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAAOeZmfDOAKBAAYrV4ihrCbsVgL79ZLuTpbg== 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.8] 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAAOeZmitpxNRAAUIZZlxi+pkN2Iy8+k+X8fQ== 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.12] 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQABOeZmqRnaBhAAFG/5Gp7PAc7jtN31aPXvdg== 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.1] 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAEOeZmLXIwHhAAvK93eEU04mG8m6v86JJ81w== 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.5] 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAEOeZmCJrbMxAAvHVAmDW6nVOcrqI337msXw== 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.9] 2024-09-15T01:31:59.475 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAFOeZmVXx6ChAAOJjfSnplmOpdURgEXtfW+g== 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.13] 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAFOeZmL7v8GhAA6bBw8aYy/6jjnln3h9yr/Q== 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.2] 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAIOeZmIDeaNBAAs2UHTFr0xO65mu0He+5I7Q== 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.6] 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAJOeZmi070CBAApu6D1pPV946BlmaCKLieag== 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.10] 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAJOeZmHW0yGhAAyVcSN6fsP+38isadzo2VzA== 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.14] 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAJOeZmzF/YKxAAqGM9VUnEXwHVtFy4Fq6gEQ== 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.x] 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQD/OOZmUV/VCRAAArjdChxqFa363iy+INXmmQ== 2024-09-15T01:31:59.476 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.3] 2024-09-15T01:31:59.477 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQANOeZmpW0/CxAAoPtyi9oc8T0V2K53ErhJtA== 2024-09-15T01:31:59.477 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.7] 2024-09-15T01:31:59.477 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQANOeZmyjGqHhAAM2eUIgH5Xd4HxjODEKKC7Q== 2024-09-15T01:31:59.477 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.11] 2024-09-15T01:31:59.477 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQANOeZmtdC0MBAAOD+I7kAS9mn+xyntNOZI5g== 2024-09-15T01:31:59.477 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.15] 2024-09-15T01:31:59.477 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAOOeZmnPu/BRAA85tgq1GT8qvxZx5Pw+kGxQ== 2024-09-15T01:31:59.477 INFO:teuthology.orchestra.run.smithi115.stdout:[client.0] 2024-09-15T01:31:59.477 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQD/OOZmeTUrDBAA+wrChYXhqHdtZoGm/2q/ug== 2024-09-15T01:31:59.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T01:31:59.517 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T01:31:59.520 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T01:31:59.570 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.572 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.575 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.623 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.625 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.627 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.673 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.676 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.678 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.727 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.773 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.776 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.778 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.826 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.828 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.874 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.877 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.879 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.925 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.927 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.929 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.975 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.977 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:59.979 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.025 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.028 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.030 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.076 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.081 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.127 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.131 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.134 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.181 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T01:32:00.184 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T01:32:00.187 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T01:32:00.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.236 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.238 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.285 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.287 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.290 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.336 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.338 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.340 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.386 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.389 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.391 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:32:00.437 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T01:32:00.439 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T01:32:00.442 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T01:32:00.490 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T01:32:00.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T01:32:00.516 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:32:00.609 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T01:32:00.634 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T01:32:00.659 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:32:00.752 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T01:32:00.777 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T01:32:00.802 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:32:00.893 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T01:32:00.918 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:32:00.921 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:32:00.923 DEBUG:teuthology.orchestra.run.smithi152:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:32:00.974 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T01:32:00.974 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T01:32:00.974 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T01:32:00.976 INFO:tasks.ceph.mon.a:Started 2024-09-15T01:32:00.977 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T01:32:00.977 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T01:32:00.979 INFO:tasks.ceph.mon.b:Started 2024-09-15T01:32:00.979 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T01:32:00.979 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T01:32:01.016 INFO:tasks.ceph.mon.c:Started 2024-09-15T01:32:01.016 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T01:32:01.016 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-15T01:32:01.016 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-15T01:32:01.019 INFO:tasks.ceph.mgr.y:Started 2024-09-15T01:32:01.019 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T01:32:01.019 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T01:32:01.021 INFO:tasks.ceph.mgr.x:Started 2024-09-15T01:32:01.021 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T01:32:01.021 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T01:32:07.395 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-09-15T01:32:07.407 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T01:32:07.408 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:32:07.408 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T01:32:07.434 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:32:07.434 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-15T01:32:07.440 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:07.438+0000 7f007ef3fec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T01:32:07.444 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:07.442+0000 7f000a228ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T01:32:07.469 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:32:07.469 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-09-15T01:32:07.479 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:07.478+0000 7f007ef3fec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T01:32:07.486 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:07.485+0000 7f000a228ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T01:32:07.511 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-15T01:32:07.511 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2024-09-15T01:32:07.528 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:07.526+0000 7f007ef3fec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T01:32:07.539 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:07.537+0000 7f000a228ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T01:32:07.564 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:32:07.564 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T01:32:07.595 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:32:07.595 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-15T01:32:07.660 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:07.658+0000 7f007ef3fec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T01:32:07.663 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:32:07.663 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-09-15T01:32:07.689 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:07.688+0000 7f000a228ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T01:32:07.730 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-15T01:32:07.730 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2024-09-15T01:32:07.764 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:07.763+0000 7f007ef3fec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T01:32:07.796 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:32:07.796 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T01:32:07.801 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:07.800+0000 7f000a228ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T01:32:07.827 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:32:07.828 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-15T01:32:07.897 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:32:07.897 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2024-09-15T01:32:07.965 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-15T01:32:07.965 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2024-09-15T01:32:08.032 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:32:08.033 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T01:32:08.055 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:08.053+0000 7f007ef3fec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T01:32:08.057 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:32:08.057 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-15T01:32:08.099 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:08.097+0000 7f000a228ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T01:32:08.104 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:08.103+0000 7f007ef3fec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T01:32:08.122 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:32:08.122 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2024-09-15T01:32:08.153 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:08.150+0000 7f000a228ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T01:32:08.187 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-15T01:32:08.188 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2024-09-15T01:32:08.242 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:08.241+0000 7f007ef3fec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T01:32:08.252 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new bec41583-a0a4-4712-a9ec-eec1e88fb60f 0 2024-09-15T01:32:08.290 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:08.288+0000 7f000a228ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T01:32:08.468 INFO:teuthology.orchestra.run.smithi175.stdout:0 2024-09-15T01:32:08.479 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 2cca66be-8685-4a38-abf8-c864cd5434da 1 2024-09-15T01:32:08.656 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-15T01:32:08.667 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new b8756aac-3a3e-41ba-a6ef-497063cb44df 2 2024-09-15T01:32:08.841 INFO:teuthology.orchestra.run.smithi175.stdout:2 2024-09-15T01:32:08.855 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 554875f5-74c0-4b58-80ee-6ba93748f104 3 2024-09-15T01:32:09.029 INFO:teuthology.orchestra.run.smithi175.stdout:3 2024-09-15T01:32:09.040 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 3fe2c05a-4c22-4dd7-a8c6-282155c82eb8 4 2024-09-15T01:32:09.187 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:09.185+0000 7f007ef3fec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T01:32:09.218 INFO:teuthology.orchestra.run.smithi175.stdout:4 2024-09-15T01:32:09.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new a301d1ce-3e31-4782-ade3-f9c0f1e6a95d 5 2024-09-15T01:32:09.240 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:09.238+0000 7f007ef3fec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T01:32:09.277 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:09.274+0000 7f000a228ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T01:32:09.330 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:09.327+0000 7f000a228ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T01:32:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:5 2024-09-15T01:32:09.416 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new c7d180be-21c5-4674-a9f2-10a1c7794871 6 2024-09-15T01:32:09.623 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:09.621+0000 7f007ef3fec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T01:32:09.636 INFO:teuthology.orchestra.run.smithi175.stdout:6 2024-09-15T01:32:09.647 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 4d2429ea-772f-4fb9-a421-240f1399f16c 7 2024-09-15T01:32:09.711 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:09.709+0000 7f007ef3fec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T01:32:09.717 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:09.714+0000 7f000a228ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T01:32:09.802 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:09.799+0000 7f000a228ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T01:32:09.803 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:09.801+0000 7f007ef3fec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T01:32:09.827 INFO:teuthology.orchestra.run.smithi175.stdout:7 2024-09-15T01:32:09.838 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new de7389ab-a29e-4177-9e94-484fc6a94102 8 2024-09-15T01:32:09.850 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:09.848+0000 7f007ef3fec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T01:32:09.895 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:09.892+0000 7f000a228ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T01:32:09.942 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:09.939+0000 7f000a228ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T01:32:09.949 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:09.948+0000 7f007ef3fec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T01:32:10.003 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:10.001+0000 7f007ef3fec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T01:32:10.016 INFO:teuthology.orchestra.run.smithi175.stdout:8 2024-09-15T01:32:10.027 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 4b4b44da-97d6-41f2-8fc6-57680b10aeb8 9 2024-09-15T01:32:10.042 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:10.039+0000 7f000a228ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T01:32:10.049 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:10.047+0000 7f007ef3fec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T01:32:10.096 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:10.094+0000 7f000a228ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T01:32:10.141 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:10.139+0000 7f000a228ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T01:32:10.147 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:10.145+0000 7f007ef3fec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T01:32:10.198 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:10.196+0000 7f007ef3fec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T01:32:10.224 INFO:teuthology.orchestra.run.smithi175.stdout:9 2024-09-15T01:32:10.235 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 5f285a07-9057-4575-b911-b3977acd397c 10 2024-09-15T01:32:10.240 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:10.238+0000 7f000a228ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T01:32:10.251 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:10.250+0000 7f007ef3fec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T01:32:10.291 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:10.289+0000 7f000a228ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T01:32:10.321 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:10.320+0000 7f007ef3fec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T01:32:10.345 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:10.343+0000 7f000a228ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T01:32:10.376 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:10.373+0000 7f007ef3fec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T01:32:10.415 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:10.412+0000 7f000a228ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T01:32:10.418 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:10.416+0000 7f007ef3fec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T01:32:10.420 INFO:teuthology.orchestra.run.smithi175.stdout:10 2024-09-15T01:32:10.431 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new dacc0f3f-a598-4055-92f4-d41102ceca83 11 2024-09-15T01:32:10.460 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:10.457+0000 7f000a228ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T01:32:10.504 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:10.502+0000 7f000a228ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T01:32:10.647 INFO:teuthology.orchestra.run.smithi175.stdout:11 2024-09-15T01:32:10.658 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new d218197b-d0db-410c-abff-8b849a21a555 12 2024-09-15T01:32:10.834 INFO:teuthology.orchestra.run.smithi175.stdout:12 2024-09-15T01:32:10.845 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new c9bf5faf-f2ee-482e-b903-28c7718e2063 13 2024-09-15T01:32:11.021 INFO:teuthology.orchestra.run.smithi175.stdout:13 2024-09-15T01:32:11.031 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new cc3f19cf-5a97-440b-a953-7727e6ed6a0d 14 2024-09-15T01:32:11.059 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:11.058+0000 7f007ef3fec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T01:32:11.155 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-15T01:32:11.153+0000 7f000a228ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T01:32:11.212 INFO:teuthology.orchestra.run.smithi175.stdout:14 2024-09-15T01:32:11.223 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 51290c23-e86d-4e05-9ac8-ca53d6a1eb80 15 2024-09-15T01:32:11.434 INFO:teuthology.orchestra.run.smithi175.stdout:15 2024-09-15T01:32:11.445 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T01:32:11.445 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T01:32:11.447 INFO:tasks.ceph.osd.0:Started 2024-09-15T01:32:11.448 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T01:32:11.448 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T01:32:11.450 INFO:tasks.ceph.osd.4:Started 2024-09-15T01:32:11.450 INFO:tasks.ceph.osd.8:Restarting daemon 2024-09-15T01:32:11.450 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-09-15T01:32:11.452 INFO:tasks.ceph.osd.8:Started 2024-09-15T01:32:11.452 INFO:tasks.ceph.osd.12:Restarting daemon 2024-09-15T01:32:11.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 12 2024-09-15T01:32:11.455 INFO:tasks.ceph.osd.12:Started 2024-09-15T01:32:11.455 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T01:32:11.455 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T01:32:11.457 INFO:tasks.ceph.osd.1:Started 2024-09-15T01:32:11.458 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T01:32:11.458 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T01:32:11.460 INFO:tasks.ceph.osd.5:Started 2024-09-15T01:32:11.460 INFO:tasks.ceph.osd.9:Restarting daemon 2024-09-15T01:32:11.460 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-09-15T01:32:11.462 INFO:tasks.ceph.osd.9:Started 2024-09-15T01:32:11.462 INFO:tasks.ceph.osd.13:Restarting daemon 2024-09-15T01:32:11.462 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 13 2024-09-15T01:32:11.464 INFO:tasks.ceph.osd.13:Started 2024-09-15T01:32:11.465 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T01:32:11.465 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T01:32:11.468 INFO:tasks.ceph.osd.2:Started 2024-09-15T01:32:11.468 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-15T01:32:11.468 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-15T01:32:11.470 INFO:tasks.ceph.osd.6:Started 2024-09-15T01:32:11.470 INFO:tasks.ceph.osd.10:Restarting daemon 2024-09-15T01:32:11.470 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10 2024-09-15T01:32:11.472 INFO:tasks.ceph.osd.10:Started 2024-09-15T01:32:11.472 INFO:tasks.ceph.osd.14:Restarting daemon 2024-09-15T01:32:11.473 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 14 2024-09-15T01:32:11.475 INFO:tasks.ceph.osd.14:Started 2024-09-15T01:32:11.475 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T01:32:11.475 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T01:32:11.477 INFO:tasks.ceph.osd.3:Started 2024-09-15T01:32:11.477 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-15T01:32:11.478 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-15T01:32:11.480 INFO:tasks.ceph.osd.7:Started 2024-09-15T01:32:11.480 INFO:tasks.ceph.osd.11:Restarting daemon 2024-09-15T01:32:11.480 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 11 2024-09-15T01:32:11.482 INFO:tasks.ceph.osd.11:Started 2024-09-15T01:32:11.482 INFO:tasks.ceph.osd.15:Restarting daemon 2024-09-15T01:32:11.482 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 15 2024-09-15T01:32:11.484 INFO:tasks.ceph.osd.15:Started 2024-09-15T01:32:11.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:32:11.527 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:11.525+0000 7f9776d23440 -1 Falling back to public interface 2024-09-15T01:32:11.533 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:11.530+0000 7f958260d440 -1 Falling back to public interface 2024-09-15T01:32:11.536 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:11.533+0000 7f044ba12440 -1 Falling back to public interface 2024-09-15T01:32:11.541 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:11.538+0000 7f9776d23440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.541 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:11.538+0000 7f9776d23440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.541 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:11.538+0000 7f9776d23440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.543 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:11.541+0000 7fd0fab23440 -1 Falling back to public interface 2024-09-15T01:32:11.545 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:11.543+0000 7f17db418440 -1 Falling back to public interface 2024-09-15T01:32:11.546 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:11.544+0000 7ff0ff247440 -1 Falling back to public interface 2024-09-15T01:32:11.548 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:11.545+0000 7f65af85a440 -1 Falling back to public interface 2024-09-15T01:32:11.549 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:11.547+0000 7f9776d23440 -1 osd.4 0 log_to_monitors true 2024-09-15T01:32:11.550 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:11.547+0000 7f958260d440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.550 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:11.547+0000 7f958260d440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.550 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:11.547+0000 7f958260d440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.551 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:11.549+0000 7f6bde00c440 -1 Falling back to public interface 2024-09-15T01:32:11.551 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:11.550+0000 7f399e84d440 -1 Falling back to public interface 2024-09-15T01:32:11.552 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:11.551+0000 7f1a43812440 -1 Falling back to public interface 2024-09-15T01:32:11.553 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:11.552+0000 7ff45391d440 -1 Falling back to public interface 2024-09-15T01:32:11.554 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:11.552+0000 7f044ba12440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.554 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:11.552+0000 7f044ba12440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.554 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:11.552+0000 7f044ba12440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.556 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:11.554+0000 7f2fe7a75440 -1 Falling back to public interface 2024-09-15T01:32:11.556 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:11.554+0000 7fd0fab23440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.556 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:11.554+0000 7fd0fab23440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.556 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:11.554+0000 7fd0fab23440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.557 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:11.556+0000 7fa505275440 -1 Falling back to public interface 2024-09-15T01:32:11.558 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:11.556+0000 7f6308921440 -1 Falling back to public interface 2024-09-15T01:32:11.559 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:11.557+0000 7f17db418440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.559 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:11.557+0000 7f17db418440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.559 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:11.557+0000 7f17db418440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.560 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:11.559+0000 7f5702419440 -1 Falling back to public interface 2024-09-15T01:32:11.561 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:11.559+0000 7f06e882f440 -1 Falling back to public interface 2024-09-15T01:32:11.564 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:11.561+0000 7fd0fab23440 -1 osd.5 0 log_to_monitors true 2024-09-15T01:32:11.564 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:11.562+0000 7f17db418440 -1 osd.1 0 log_to_monitors true 2024-09-15T01:32:11.565 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:11.562+0000 7f958260d440 -1 osd.0 0 log_to_monitors true 2024-09-15T01:32:11.565 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:11.563+0000 7ff45391d440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.565 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:11.563+0000 7ff45391d440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.565 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:11.563+0000 7ff45391d440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.566 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:11.563+0000 7ff0ff247440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.566 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:11.563+0000 7ff0ff247440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.566 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:11.563+0000 7ff0ff247440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.567 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:11.565+0000 7f1a43812440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.567 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:11.565+0000 7f1a43812440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.567 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:11.565+0000 7f1a43812440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.567 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:11.565+0000 7f399e84d440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.567 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:11.565+0000 7f399e84d440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.568 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:11.565+0000 7f399e84d440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.568 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:11.565+0000 7f044ba12440 -1 osd.8 0 log_to_monitors true 2024-09-15T01:32:11.569 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:11.566+0000 7f2fe7a75440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.569 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:11.566+0000 7f2fe7a75440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.569 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:11.566+0000 7f2fe7a75440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.570 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:11.567+0000 7f65af85a440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.570 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:11.567+0000 7f65af85a440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.570 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:11.567+0000 7f65af85a440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.571 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:11.568+0000 7f6bde00c440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.571 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:11.568+0000 7f6bde00c440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.571 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:11.568+0000 7f6bde00c440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.572 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:11.570+0000 7f399e84d440 -1 osd.3 0 log_to_monitors true 2024-09-15T01:32:11.572 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:11.570+0000 7f1a43812440 -1 osd.7 0 log_to_monitors true 2024-09-15T01:32:11.572 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:11.570+0000 7ff45391d440 -1 osd.6 0 log_to_monitors true 2024-09-15T01:32:11.573 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:11.571+0000 7ff0ff247440 -1 osd.9 0 log_to_monitors true 2024-09-15T01:32:11.575 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:11.573+0000 7fa505275440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.575 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:11.573+0000 7fa505275440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.575 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:11.573+0000 7fa505275440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.575 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:11.573+0000 7f2fe7a75440 -1 osd.2 0 log_to_monitors true 2024-09-15T01:32:11.576 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:11.574+0000 7f5702419440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.576 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:11.574+0000 7f5702419440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.576 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:11.574+0000 7f5702419440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.577 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:11.575+0000 7f6308921440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.577 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:11.575+0000 7f6308921440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.577 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:11.575+0000 7f6308921440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.579 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:11.577+0000 7f06e882f440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:32:11.579 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:11.577+0000 7f6bde00c440 -1 osd.13 0 log_to_monitors true 2024-09-15T01:32:11.579 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:11.576+0000 7f65af85a440 -1 osd.12 0 log_to_monitors true 2024-09-15T01:32:11.580 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:11.578+0000 7fa505275440 -1 osd.15 0 log_to_monitors true 2024-09-15T01:32:11.580 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:11.578+0000 7f06e882f440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.580 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:11.578+0000 7f06e882f440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:32:11.583 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:11.581+0000 7f5702419440 -1 osd.11 0 log_to_monitors true 2024-09-15T01:32:11.583 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:11.582+0000 7f6308921440 -1 osd.10 0 log_to_monitors true 2024-09-15T01:32:11.584 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:11.583+0000 7f06e882f440 -1 osd.14 0 log_to_monitors true 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":18,"fsid":"d4803586-ed44-4154-8ee1-6b95ec3e17e7","created":"2024-09-15T01:32:07.285177+0000","modified":"2024-09-15T01:32:11.430401+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T01:32:11.430401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"bec41583-a0a4-4712-a9ec-eec1e88fb60f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2cca66be-8685-4a38-abf8-c864cd5434da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b8756aac-3a3e-41ba-a6ef-497063cb44df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"554875f5-74c0-4b58-80ee-6ba93748f104","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3fe2c05a-4c22-4dd7-a8c6-282155c82eb8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a301d1ce-3e31-4782-ade3-f9c0f1e6a95d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c7d180be-21c5-4674-a9f2-10a1c7794871","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4d2429ea-772f-4fb9-a421-240f1399f16c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"de7389ab-a29e-4177-9e94-484fc6a94102","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"4b4b44da-97d6-41f2-8fc6-57680b10aeb8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"5f285a07-9057-4575-b911-b3977acd397c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"dacc0f3f-a598-4055-92f4-d41102ceca83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"d218197b-d0db-410c-abff-8b849a21a555","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c9bf5faf-f2ee-482e-b903-28c7718e2063","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cc3f19cf-5a97-440b-a953-7727e6ed6a0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"51290c23-e86d-4e05-9ac8-ca53d6a1eb80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:32:11.689 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T01:32:11.689 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T01:32:12.078 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:12.076+0000 7f0063e5b640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T01:32:12.081 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-15T01:32:12.079+0000 7f0063e5b640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T01:32:13.303 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T01:32:13.443 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:13.441+0000 7f6303bf7640 -1 osd.10 0 waiting for initial osdmap 2024-09-15T01:32:13.445 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:13.442+0000 7fd0f5df7640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T01:32:13.445 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:13.442+0000 7f04469f7640 -1 osd.8 0 waiting for initial osdmap 2024-09-15T01:32:13.446 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:13.443+0000 7f957d5f7640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T01:32:13.446 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:13.444+0000 7f65ab1f7640 -1 osd.12 0 waiting for initial osdmap 2024-09-15T01:32:13.447 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:13.445+0000 7ff44ebf7640 -1 osd.6 0 waiting for initial osdmap 2024-09-15T01:32:13.447 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:13.445+0000 7f2fe33f7640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T01:32:13.447 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:13.445+0000 7f63003f0640 -1 osd.10 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.447 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:13.445+0000 7f39997f7640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T01:32:13.448 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:13.446+0000 7fd0f25f0640 -1 osd.5 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.453 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:13.450+0000 7f17d63f7640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T01:32:13.453 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:13.450+0000 7f9771ff7640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T01:32:13.453 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:13.450+0000 7ff0fa1f7640 -1 osd.9 0 waiting for initial osdmap 2024-09-15T01:32:13.453 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:13.450+0000 7f6bd8ff7640 -1 osd.13 0 waiting for initial osdmap 2024-09-15T01:32:13.454 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:13.451+0000 7f06e41f7640 -1 osd.14 0 waiting for initial osdmap 2024-09-15T01:32:13.454 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:13.451+0000 7f1a3e7f7640 -1 osd.7 0 waiting for initial osdmap 2024-09-15T01:32:13.454 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:13.451+0000 7f04431f0640 -1 osd.8 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.455 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:13.451+0000 7fa500bf7640 -1 osd.15 0 waiting for initial osdmap 2024-09-15T01:32:13.455 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:13.451+0000 7f65a788c640 -1 osd.12 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.455 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:13.451+0000 7f56fd3f7640 -1 osd.11 0 waiting for initial osdmap 2024-09-15T01:32:13.455 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:13.451+0000 7f9579c8c640 -1 osd.0 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.456 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:13.452+0000 7f3995ff0640 -1 osd.3 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.456 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:13.453+0000 7f976e7f0640 -1 osd.4 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.456 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:13.453+0000 7ff0f69f0640 -1 osd.9 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.457 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:13.453+0000 7f2fdfa8c640 -1 osd.2 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.457 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:13.453+0000 7f17d2bf0640 -1 osd.1 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.457 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:13.453+0000 7ff44b3f0640 -1 osd.6 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.457 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:13.453+0000 7f6bd57f0640 -1 osd.13 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.457 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:13.455+0000 7f06e088c640 -1 osd.14 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.458 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:13.455+0000 7f1a3aff0640 -1 osd.7 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.458 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:13.455+0000 7fa4fd28c640 -1 osd.15 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.458 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:13.456+0000 7f56f9bf0640 -1 osd.11 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:32:13.601 INFO:teuthology.misc.health.smithi042.stdout: 2024-09-15T01:32:13.601 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":20,"fsid":"d4803586-ed44-4154-8ee1-6b95ec3e17e7","created":"2024-09-15T01:32:07.285177+0000","modified":"2024-09-15T01:32:13.432957+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T01:32:11.430401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"bec41583-a0a4-4712-a9ec-eec1e88fb60f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2cca66be-8685-4a38-abf8-c864cd5434da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b8756aac-3a3e-41ba-a6ef-497063cb44df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"554875f5-74c0-4b58-80ee-6ba93748f104","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3fe2c05a-4c22-4dd7-a8c6-282155c82eb8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a301d1ce-3e31-4782-ade3-f9c0f1e6a95d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c7d180be-21c5-4674-a9f2-10a1c7794871","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4d2429ea-772f-4fb9-a421-240f1399f16c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"de7389ab-a29e-4177-9e94-484fc6a94102","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"4b4b44da-97d6-41f2-8fc6-57680b10aeb8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"5f285a07-9057-4575-b911-b3977acd397c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"dacc0f3f-a598-4055-92f4-d41102ceca83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"d218197b-d0db-410c-abff-8b849a21a555","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c9bf5faf-f2ee-482e-b903-28c7718e2063","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cc3f19cf-5a97-440b-a953-7727e6ed6a0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"51290c23-e86d-4e05-9ac8-ca53d6a1eb80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:32:13.612 DEBUG:teuthology.misc:0 of 16 OSDs are up 2024-09-15T01:32:21.227 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T01:32:21.510 INFO:teuthology.misc.health.smithi042.stdout: 2024-09-15T01:32:21.510 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":24,"fsid":"d4803586-ed44-4154-8ee1-6b95ec3e17e7","created":"2024-09-15T01:32:07.285177+0000","modified":"2024-09-15T01:32:17.465122+0000","last_up_change":"2024-09-15T01:32:14.443323+0000","last_in_change":"2024-09-15T01:32:11.430401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:32:15.118634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"bec41583-a0a4-4712-a9ec-eec1e88fb60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":28861},{"type":"v1","addr":"172.21.15.42:6809","nonce":28861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":28861},{"type":"v1","addr":"172.21.15.42:6811","nonce":28861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":28861},{"type":"v1","addr":"172.21.15.42:6815","nonce":28861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":28861},{"type":"v1","addr":"172.21.15.42:6813","nonce":28861}]},"public_addr":"172.21.15.42:6809/28861","cluster_addr":"172.21.15.42:6811/28861","heartbeat_back_addr":"172.21.15.42:6815/28861","heartbeat_front_addr":"172.21.15.42:6813/28861","state":["exists","up"]},{"osd":1,"uuid":"2cca66be-8685-4a38-abf8-c864cd5434da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":28508},{"type":"v1","addr":"172.21.15.115:6809","nonce":28508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":28508},{"type":"v1","addr":"172.21.15.115:6811","nonce":28508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6817","nonce":28508},{"type":"v1","addr":"172.21.15.115:6819","nonce":28508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6813","nonce":28508},{"type":"v1","addr":"172.21.15.115:6815","nonce":28508}]},"public_addr":"172.21.15.115:6809/28508","cluster_addr":"172.21.15.115:6811/28508","heartbeat_back_addr":"172.21.15.115:6819/28508","heartbeat_front_addr":"172.21.15.115:6815/28508","state":["exists","up"]},{"osd":2,"uuid":"b8756aac-3a3e-41ba-a6ef-497063cb44df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":28732},{"type":"v1","addr":"172.21.15.152:6809","nonce":28732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":28732},{"type":"v1","addr":"172.21.15.152:6811","nonce":28732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":28732},{"type":"v1","addr":"172.21.15.152:6816","nonce":28732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":28732},{"type":"v1","addr":"172.21.15.152:6813","nonce":28732}]},"public_addr":"172.21.15.152:6809/28732","cluster_addr":"172.21.15.152:6811/28732","heartbeat_back_addr":"172.21.15.152:6816/28732","heartbeat_front_addr":"172.21.15.152:6813/28732","state":["exists","up"]},{"osd":3,"uuid":"554875f5-74c0-4b58-80ee-6ba93748f104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":28835},{"type":"v1","addr":"172.21.15.175:6803","nonce":28835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":28835},{"type":"v1","addr":"172.21.15.175:6805","nonce":28835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":28835},{"type":"v1","addr":"172.21.15.175:6809","nonce":28835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":28835},{"type":"v1","addr":"172.21.15.175:6807","nonce":28835}]},"public_addr":"172.21.15.175:6803/28835","cluster_addr":"172.21.15.175:6805/28835","heartbeat_back_addr":"172.21.15.175:6809/28835","heartbeat_front_addr":"172.21.15.175:6807/28835","state":["exists","up"]},{"osd":4,"uuid":"3fe2c05a-4c22-4dd7-a8c6-282155c82eb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":28860},{"type":"v1","addr":"172.21.15.42:6801","nonce":28860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":28860},{"type":"v1","addr":"172.21.15.42:6803","nonce":28860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":28860},{"type":"v1","addr":"172.21.15.42:6807","nonce":28860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":28860},{"type":"v1","addr":"172.21.15.42:6805","nonce":28860}]},"public_addr":"172.21.15.42:6801/28860","cluster_addr":"172.21.15.42:6803/28860","heartbeat_back_addr":"172.21.15.42:6807/28860","heartbeat_front_addr":"172.21.15.42:6805/28860","state":["exists","up"]},{"osd":5,"uuid":"a301d1ce-3e31-4782-ade3-f9c0f1e6a95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":28509},{"type":"v1","addr":"172.21.15.115:6801","nonce":28509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":28509},{"type":"v1","addr":"172.21.15.115:6803","nonce":28509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":28509},{"type":"v1","addr":"172.21.15.115:6807","nonce":28509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":28509},{"type":"v1","addr":"172.21.15.115:6805","nonce":28509}]},"public_addr":"172.21.15.115:6801/28509","cluster_addr":"172.21.15.115:6803/28509","heartbeat_back_addr":"172.21.15.115:6807/28509","heartbeat_front_addr":"172.21.15.115:6805/28509","state":["exists","up"]},{"osd":6,"uuid":"c7d180be-21c5-4674-a9f2-10a1c7794871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":28731},{"type":"v1","addr":"172.21.15.152:6801","nonce":28731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":28731},{"type":"v1","addr":"172.21.15.152:6803","nonce":28731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":28731},{"type":"v1","addr":"172.21.15.152:6807","nonce":28731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":28731},{"type":"v1","addr":"172.21.15.152:6805","nonce":28731}]},"public_addr":"172.21.15.152:6801/28731","cluster_addr":"172.21.15.152:6803/28731","heartbeat_back_addr":"172.21.15.152:6807/28731","heartbeat_front_addr":"172.21.15.152:6805/28731","state":["exists","up"]},{"osd":7,"uuid":"4d2429ea-772f-4fb9-a421-240f1399f16c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":28836},{"type":"v1","addr":"172.21.15.175:6811","nonce":28836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":28836},{"type":"v1","addr":"172.21.15.175:6813","nonce":28836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":28836},{"type":"v1","addr":"172.21.15.175:6817","nonce":28836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":28836},{"type":"v1","addr":"172.21.15.175:6815","nonce":28836}]},"public_addr":"172.21.15.175:6811/28836","cluster_addr":"172.21.15.175:6813/28836","heartbeat_back_addr":"172.21.15.175:6817/28836","heartbeat_front_addr":"172.21.15.175:6815/28836","state":["exists","up"]},{"osd":8,"uuid":"de7389ab-a29e-4177-9e94-484fc6a94102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":28862},{"type":"v1","addr":"172.21.15.42:6817","nonce":28862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":28862},{"type":"v1","addr":"172.21.15.42:6819","nonce":28862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":28862},{"type":"v1","addr":"172.21.15.42:6823","nonce":28862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":28862},{"type":"v1","addr":"172.21.15.42:6821","nonce":28862}]},"public_addr":"172.21.15.42:6817/28862","cluster_addr":"172.21.15.42:6819/28862","heartbeat_back_addr":"172.21.15.42:6823/28862","heartbeat_front_addr":"172.21.15.42:6821/28862","state":["exists","up"]},{"osd":9,"uuid":"4b4b44da-97d6-41f2-8fc6-57680b10aeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":28510},{"type":"v1","addr":"172.21.15.115:6814","nonce":28510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":28510},{"type":"v1","addr":"172.21.15.115:6818","nonce":28510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":28510},{"type":"v1","addr":"172.21.15.115:6823","nonce":28510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":28510},{"type":"v1","addr":"172.21.15.115:6821","nonce":28510}]},"public_addr":"172.21.15.115:6814/28510","cluster_addr":"172.21.15.115:6818/28510","heartbeat_back_addr":"172.21.15.115:6823/28510","heartbeat_front_addr":"172.21.15.115:6821/28510","state":["exists","up"]},{"osd":10,"uuid":"5f285a07-9057-4575-b911-b3977acd397c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":28733},{"type":"v1","addr":"172.21.15.152:6817","nonce":28733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":28733},{"type":"v1","addr":"172.21.15.152:6819","nonce":28733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":28733},{"type":"v1","addr":"172.21.15.152:6824","nonce":28733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":28733},{"type":"v1","addr":"172.21.15.152:6821","nonce":28733}]},"public_addr":"172.21.15.152:6817/28733","cluster_addr":"172.21.15.152:6819/28733","heartbeat_back_addr":"172.21.15.152:6824/28733","heartbeat_front_addr":"172.21.15.152:6821/28733","state":["exists","up"]},{"osd":11,"uuid":"dacc0f3f-a598-4055-92f4-d41102ceca83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":28837},{"type":"v1","addr":"172.21.15.175:6827","nonce":28837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":28837},{"type":"v1","addr":"172.21.15.175:6829","nonce":28837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":28837},{"type":"v1","addr":"172.21.15.175:6833","nonce":28837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":28837},{"type":"v1","addr":"172.21.15.175:6831","nonce":28837}]},"public_addr":"172.21.15.175:6827/28837","cluster_addr":"172.21.15.175:6829/28837","heartbeat_back_addr":"172.21.15.175:6833/28837","heartbeat_front_addr":"172.21.15.175:6831/28837","state":["exists","up"]},{"osd":12,"uuid":"d218197b-d0db-410c-abff-8b849a21a555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":28866},{"type":"v1","addr":"172.21.15.42:6825","nonce":28866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":28866},{"type":"v1","addr":"172.21.15.42:6827","nonce":28866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":28866},{"type":"v1","addr":"172.21.15.42:6831","nonce":28866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":28866},{"type":"v1","addr":"172.21.15.42:6829","nonce":28866}]},"public_addr":"172.21.15.42:6825/28866","cluster_addr":"172.21.15.42:6827/28866","heartbeat_back_addr":"172.21.15.42:6831/28866","heartbeat_front_addr":"172.21.15.42:6829/28866","state":["exists","up"]},{"osd":13,"uuid":"c9bf5faf-f2ee-482e-b903-28c7718e2063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":28511},{"type":"v1","addr":"172.21.15.115:6825","nonce":28511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":28511},{"type":"v1","addr":"172.21.15.115:6827","nonce":28511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":28511},{"type":"v1","addr":"172.21.15.115:6831","nonce":28511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":28511},{"type":"v1","addr":"172.21.15.115:6829","nonce":28511}]},"public_addr":"172.21.15.115:6825/28511","cluster_addr":"172.21.15.115:6827/28511","heartbeat_back_addr":"172.21.15.115:6831/28511","heartbeat_front_addr":"172.21.15.115:6829/28511","state":["exists","up"]},{"osd":14,"uuid":"cc3f19cf-5a97-440b-a953-7727e6ed6a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6823","nonce":28734},{"type":"v1","addr":"172.21.15.152:6825","nonce":28734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":28734},{"type":"v1","addr":"172.21.15.152:6827","nonce":28734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":28734},{"type":"v1","addr":"172.21.15.152:6831","nonce":28734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":28734},{"type":"v1","addr":"172.21.15.152:6829","nonce":28734}]},"public_addr":"172.21.15.152:6825/28734","cluster_addr":"172.21.15.152:6827/28734","heartbeat_back_addr":"172.21.15.152:6831/28734","heartbeat_front_addr":"172.21.15.152:6829/28734","state":["exists","up"]},{"osd":15,"uuid":"51290c23-e86d-4e05-9ac8-ca53d6a1eb80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":28838},{"type":"v1","addr":"172.21.15.175:6819","nonce":28838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":28838},{"type":"v1","addr":"172.21.15.175:6821","nonce":28838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":28838},{"type":"v1","addr":"172.21.15.175:6825","nonce":28838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":28838},{"type":"v1","addr":"172.21.15.175:6823","nonce":28838}]},"public_addr":"172.21.15.175:6819/28838","cluster_addr":"172.21.15.175:6821/28838","heartbeat_back_addr":"172.21.15.175:6825/28838","heartbeat_front_addr":"172.21.15.175:6823/28838","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.539908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.544884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.614659+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.605046+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.503545+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.584788+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.563649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.622071+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.572859+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.526526+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.534845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.556801+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.565526+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.582172+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.619730+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.619890+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:32:21.521 DEBUG:teuthology.misc:16 of 16 OSDs are up 2024-09-15T01:32:21.521 INFO:tasks.ceph:Creating RBD pool 2024-09-15T01:32:21.521 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T01:32:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'rbd' created 2024-09-15T01:32:22.142 DEBUG:teuthology.orchestra.run.smithi042:> rbd --cluster ceph pool init rbd 2024-09-15T01:32:25.152 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T01:32:25.152 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T01:32:25.152 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T01:32:25.152 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T01:32:25.153 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T01:32:25.486 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:25.496 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":27931},{"type":"v1","addr":"172.21.15.175:6801","nonce":27931}]},"active_addr":"172.21.15.175:6801/27931","active_change":"2024-09-15T01:32:11.061422+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":4179064839}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":3658765257}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":4051298523}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":4045615310}]}]} 2024-09-15T01:32:25.501 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T01:32:25.501 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T01:32:25.502 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:32:25.780 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:25.780 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":28,"fsid":"d4803586-ed44-4154-8ee1-6b95ec3e17e7","created":"2024-09-15T01:32:07.285177+0000","modified":"2024-09-15T01:32:25.135277+0000","last_up_change":"2024-09-15T01:32:14.443323+0000","last_in_change":"2024-09-15T01:32:11.430401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:32:15.118634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:32:21.824131+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bec41583-a0a4-4712-a9ec-eec1e88fb60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":28861},{"type":"v1","addr":"172.21.15.42:6809","nonce":28861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":28861},{"type":"v1","addr":"172.21.15.42:6811","nonce":28861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":28861},{"type":"v1","addr":"172.21.15.42:6815","nonce":28861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":28861},{"type":"v1","addr":"172.21.15.42:6813","nonce":28861}]},"public_addr":"172.21.15.42:6809/28861","cluster_addr":"172.21.15.42:6811/28861","heartbeat_back_addr":"172.21.15.42:6815/28861","heartbeat_front_addr":"172.21.15.42:6813/28861","state":["exists","up"]},{"osd":1,"uuid":"2cca66be-8685-4a38-abf8-c864cd5434da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":28508},{"type":"v1","addr":"172.21.15.115:6809","nonce":28508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":28508},{"type":"v1","addr":"172.21.15.115:6811","nonce":28508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6817","nonce":28508},{"type":"v1","addr":"172.21.15.115:6819","nonce":28508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6813","nonce":28508},{"type":"v1","addr":"172.21.15.115:6815","nonce":28508}]},"public_addr":"172.21.15.115:6809/28508","cluster_addr":"172.21.15.115:6811/28508","heartbeat_back_addr":"172.21.15.115:6819/28508","heartbeat_front_addr":"172.21.15.115:6815/28508","state":["exists","up"]},{"osd":2,"uuid":"b8756aac-3a3e-41ba-a6ef-497063cb44df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":28732},{"type":"v1","addr":"172.21.15.152:6809","nonce":28732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":28732},{"type":"v1","addr":"172.21.15.152:6811","nonce":28732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":28732},{"type":"v1","addr":"172.21.15.152:6816","nonce":28732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":28732},{"type":"v1","addr":"172.21.15.152:6813","nonce":28732}]},"public_addr":"172.21.15.152:6809/28732","cluster_addr":"172.21.15.152:6811/28732","heartbeat_back_addr":"172.21.15.152:6816/28732","heartbeat_front_addr":"172.21.15.152:6813/28732","state":["exists","up"]},{"osd":3,"uuid":"554875f5-74c0-4b58-80ee-6ba93748f104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":28835},{"type":"v1","addr":"172.21.15.175:6803","nonce":28835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":28835},{"type":"v1","addr":"172.21.15.175:6805","nonce":28835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":28835},{"type":"v1","addr":"172.21.15.175:6809","nonce":28835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":28835},{"type":"v1","addr":"172.21.15.175:6807","nonce":28835}]},"public_addr":"172.21.15.175:6803/28835","cluster_addr":"172.21.15.175:6805/28835","heartbeat_back_addr":"172.21.15.175:6809/28835","heartbeat_front_addr":"172.21.15.175:6807/28835","state":["exists","up"]},{"osd":4,"uuid":"3fe2c05a-4c22-4dd7-a8c6-282155c82eb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":28860},{"type":"v1","addr":"172.21.15.42:6801","nonce":28860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":28860},{"type":"v1","addr":"172.21.15.42:6803","nonce":28860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":28860},{"type":"v1","addr":"172.21.15.42:6807","nonce":28860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":28860},{"type":"v1","addr":"172.21.15.42:6805","nonce":28860}]},"public_addr":"172.21.15.42:6801/28860","cluster_addr":"172.21.15.42:6803/28860","heartbeat_back_addr":"172.21.15.42:6807/28860","heartbeat_front_addr":"172.21.15.42:6805/28860","state":["exists","up"]},{"osd":5,"uuid":"a301d1ce-3e31-4782-ade3-f9c0f1e6a95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":28509},{"type":"v1","addr":"172.21.15.115:6801","nonce":28509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":28509},{"type":"v1","addr":"172.21.15.115:6803","nonce":28509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":28509},{"type":"v1","addr":"172.21.15.115:6807","nonce":28509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":28509},{"type":"v1","addr":"172.21.15.115:6805","nonce":28509}]},"public_addr":"172.21.15.115:6801/28509","cluster_addr":"172.21.15.115:6803/28509","heartbeat_back_addr":"172.21.15.115:6807/28509","heartbeat_front_addr":"172.21.15.115:6805/28509","state":["exists","up"]},{"osd":6,"uuid":"c7d180be-21c5-4674-a9f2-10a1c7794871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":28731},{"type":"v1","addr":"172.21.15.152:6801","nonce":28731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":28731},{"type":"v1","addr":"172.21.15.152:6803","nonce":28731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":28731},{"type":"v1","addr":"172.21.15.152:6807","nonce":28731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":28731},{"type":"v1","addr":"172.21.15.152:6805","nonce":28731}]},"public_addr":"172.21.15.152:6801/28731","cluster_addr":"172.21.15.152:6803/28731","heartbeat_back_addr":"172.21.15.152:6807/28731","heartbeat_front_addr":"172.21.15.152:6805/28731","state":["exists","up"]},{"osd":7,"uuid":"4d2429ea-772f-4fb9-a421-240f1399f16c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":28836},{"type":"v1","addr":"172.21.15.175:6811","nonce":28836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":28836},{"type":"v1","addr":"172.21.15.175:6813","nonce":28836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":28836},{"type":"v1","addr":"172.21.15.175:6817","nonce":28836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":28836},{"type":"v1","addr":"172.21.15.175:6815","nonce":28836}]},"public_addr":"172.21.15.175:6811/28836","cluster_addr":"172.21.15.175:6813/28836","heartbeat_back_addr":"172.21.15.175:6817/28836","heartbeat_front_addr":"172.21.15.175:6815/28836","state":["exists","up"]},{"osd":8,"uuid":"de7389ab-a29e-4177-9e94-484fc6a94102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":28862},{"type":"v1","addr":"172.21.15.42:6817","nonce":28862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":28862},{"type":"v1","addr":"172.21.15.42:6819","nonce":28862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":28862},{"type":"v1","addr":"172.21.15.42:6823","nonce":28862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":28862},{"type":"v1","addr":"172.21.15.42:6821","nonce":28862}]},"public_addr":"172.21.15.42:6817/28862","cluster_addr":"172.21.15.42:6819/28862","heartbeat_back_addr":"172.21.15.42:6823/28862","heartbeat_front_addr":"172.21.15.42:6821/28862","state":["exists","up"]},{"osd":9,"uuid":"4b4b44da-97d6-41f2-8fc6-57680b10aeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":28510},{"type":"v1","addr":"172.21.15.115:6814","nonce":28510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":28510},{"type":"v1","addr":"172.21.15.115:6818","nonce":28510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":28510},{"type":"v1","addr":"172.21.15.115:6823","nonce":28510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":28510},{"type":"v1","addr":"172.21.15.115:6821","nonce":28510}]},"public_addr":"172.21.15.115:6814/28510","cluster_addr":"172.21.15.115:6818/28510","heartbeat_back_addr":"172.21.15.115:6823/28510","heartbeat_front_addr":"172.21.15.115:6821/28510","state":["exists","up"]},{"osd":10,"uuid":"5f285a07-9057-4575-b911-b3977acd397c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":28733},{"type":"v1","addr":"172.21.15.152:6817","nonce":28733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":28733},{"type":"v1","addr":"172.21.15.152:6819","nonce":28733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":28733},{"type":"v1","addr":"172.21.15.152:6824","nonce":28733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":28733},{"type":"v1","addr":"172.21.15.152:6821","nonce":28733}]},"public_addr":"172.21.15.152:6817/28733","cluster_addr":"172.21.15.152:6819/28733","heartbeat_back_addr":"172.21.15.152:6824/28733","heartbeat_front_addr":"172.21.15.152:6821/28733","state":["exists","up"]},{"osd":11,"uuid":"dacc0f3f-a598-4055-92f4-d41102ceca83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":28837},{"type":"v1","addr":"172.21.15.175:6827","nonce":28837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":28837},{"type":"v1","addr":"172.21.15.175:6829","nonce":28837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":28837},{"type":"v1","addr":"172.21.15.175:6833","nonce":28837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":28837},{"type":"v1","addr":"172.21.15.175:6831","nonce":28837}]},"public_addr":"172.21.15.175:6827/28837","cluster_addr":"172.21.15.175:6829/28837","heartbeat_back_addr":"172.21.15.175:6833/28837","heartbeat_front_addr":"172.21.15.175:6831/28837","state":["exists","up"]},{"osd":12,"uuid":"d218197b-d0db-410c-abff-8b849a21a555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":28866},{"type":"v1","addr":"172.21.15.42:6825","nonce":28866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":28866},{"type":"v1","addr":"172.21.15.42:6827","nonce":28866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":28866},{"type":"v1","addr":"172.21.15.42:6831","nonce":28866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":28866},{"type":"v1","addr":"172.21.15.42:6829","nonce":28866}]},"public_addr":"172.21.15.42:6825/28866","cluster_addr":"172.21.15.42:6827/28866","heartbeat_back_addr":"172.21.15.42:6831/28866","heartbeat_front_addr":"172.21.15.42:6829/28866","state":["exists","up"]},{"osd":13,"uuid":"c9bf5faf-f2ee-482e-b903-28c7718e2063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":28511},{"type":"v1","addr":"172.21.15.115:6825","nonce":28511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":28511},{"type":"v1","addr":"172.21.15.115:6827","nonce":28511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":28511},{"type":"v1","addr":"172.21.15.115:6831","nonce":28511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":28511},{"type":"v1","addr":"172.21.15.115:6829","nonce":28511}]},"public_addr":"172.21.15.115:6825/28511","cluster_addr":"172.21.15.115:6827/28511","heartbeat_back_addr":"172.21.15.115:6831/28511","heartbeat_front_addr":"172.21.15.115:6829/28511","state":["exists","up"]},{"osd":14,"uuid":"cc3f19cf-5a97-440b-a953-7727e6ed6a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6823","nonce":28734},{"type":"v1","addr":"172.21.15.152:6825","nonce":28734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":28734},{"type":"v1","addr":"172.21.15.152:6827","nonce":28734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":28734},{"type":"v1","addr":"172.21.15.152:6831","nonce":28734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":28734},{"type":"v1","addr":"172.21.15.152:6829","nonce":28734}]},"public_addr":"172.21.15.152:6825/28734","cluster_addr":"172.21.15.152:6827/28734","heartbeat_back_addr":"172.21.15.152:6831/28734","heartbeat_front_addr":"172.21.15.152:6829/28734","state":["exists","up"]},{"osd":15,"uuid":"51290c23-e86d-4e05-9ac8-ca53d6a1eb80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":28838},{"type":"v1","addr":"172.21.15.175:6819","nonce":28838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":28838},{"type":"v1","addr":"172.21.15.175:6821","nonce":28838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":28838},{"type":"v1","addr":"172.21.15.175:6825","nonce":28838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":28838},{"type":"v1","addr":"172.21.15.175:6823","nonce":28838}]},"public_addr":"172.21.15.175:6819/28838","cluster_addr":"172.21.15.175:6821/28838","heartbeat_back_addr":"172.21.15.175:6825/28838","heartbeat_front_addr":"172.21.15.175:6823/28838","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.539908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.544884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.614659+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.605046+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.503545+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.584788+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.563649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.622071+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.572859+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.526526+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.534845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.556801+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.565526+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.582172+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.619730+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.619890+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:32:25.792 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T01:32:25.793 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:32:26.070 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:26.071 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":28,"fsid":"d4803586-ed44-4154-8ee1-6b95ec3e17e7","created":"2024-09-15T01:32:07.285177+0000","modified":"2024-09-15T01:32:25.135277+0000","last_up_change":"2024-09-15T01:32:14.443323+0000","last_in_change":"2024-09-15T01:32:11.430401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:32:15.118634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:32:21.824131+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bec41583-a0a4-4712-a9ec-eec1e88fb60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":28861},{"type":"v1","addr":"172.21.15.42:6809","nonce":28861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":28861},{"type":"v1","addr":"172.21.15.42:6811","nonce":28861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":28861},{"type":"v1","addr":"172.21.15.42:6815","nonce":28861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":28861},{"type":"v1","addr":"172.21.15.42:6813","nonce":28861}]},"public_addr":"172.21.15.42:6809/28861","cluster_addr":"172.21.15.42:6811/28861","heartbeat_back_addr":"172.21.15.42:6815/28861","heartbeat_front_addr":"172.21.15.42:6813/28861","state":["exists","up"]},{"osd":1,"uuid":"2cca66be-8685-4a38-abf8-c864cd5434da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":28508},{"type":"v1","addr":"172.21.15.115:6809","nonce":28508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":28508},{"type":"v1","addr":"172.21.15.115:6811","nonce":28508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6817","nonce":28508},{"type":"v1","addr":"172.21.15.115:6819","nonce":28508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6813","nonce":28508},{"type":"v1","addr":"172.21.15.115:6815","nonce":28508}]},"public_addr":"172.21.15.115:6809/28508","cluster_addr":"172.21.15.115:6811/28508","heartbeat_back_addr":"172.21.15.115:6819/28508","heartbeat_front_addr":"172.21.15.115:6815/28508","state":["exists","up"]},{"osd":2,"uuid":"b8756aac-3a3e-41ba-a6ef-497063cb44df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":28732},{"type":"v1","addr":"172.21.15.152:6809","nonce":28732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":28732},{"type":"v1","addr":"172.21.15.152:6811","nonce":28732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":28732},{"type":"v1","addr":"172.21.15.152:6816","nonce":28732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":28732},{"type":"v1","addr":"172.21.15.152:6813","nonce":28732}]},"public_addr":"172.21.15.152:6809/28732","cluster_addr":"172.21.15.152:6811/28732","heartbeat_back_addr":"172.21.15.152:6816/28732","heartbeat_front_addr":"172.21.15.152:6813/28732","state":["exists","up"]},{"osd":3,"uuid":"554875f5-74c0-4b58-80ee-6ba93748f104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":28835},{"type":"v1","addr":"172.21.15.175:6803","nonce":28835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":28835},{"type":"v1","addr":"172.21.15.175:6805","nonce":28835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":28835},{"type":"v1","addr":"172.21.15.175:6809","nonce":28835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":28835},{"type":"v1","addr":"172.21.15.175:6807","nonce":28835}]},"public_addr":"172.21.15.175:6803/28835","cluster_addr":"172.21.15.175:6805/28835","heartbeat_back_addr":"172.21.15.175:6809/28835","heartbeat_front_addr":"172.21.15.175:6807/28835","state":["exists","up"]},{"osd":4,"uuid":"3fe2c05a-4c22-4dd7-a8c6-282155c82eb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":28860},{"type":"v1","addr":"172.21.15.42:6801","nonce":28860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":28860},{"type":"v1","addr":"172.21.15.42:6803","nonce":28860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":28860},{"type":"v1","addr":"172.21.15.42:6807","nonce":28860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":28860},{"type":"v1","addr":"172.21.15.42:6805","nonce":28860}]},"public_addr":"172.21.15.42:6801/28860","cluster_addr":"172.21.15.42:6803/28860","heartbeat_back_addr":"172.21.15.42:6807/28860","heartbeat_front_addr":"172.21.15.42:6805/28860","state":["exists","up"]},{"osd":5,"uuid":"a301d1ce-3e31-4782-ade3-f9c0f1e6a95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":28509},{"type":"v1","addr":"172.21.15.115:6801","nonce":28509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":28509},{"type":"v1","addr":"172.21.15.115:6803","nonce":28509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":28509},{"type":"v1","addr":"172.21.15.115:6807","nonce":28509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":28509},{"type":"v1","addr":"172.21.15.115:6805","nonce":28509}]},"public_addr":"172.21.15.115:6801/28509","cluster_addr":"172.21.15.115:6803/28509","heartbeat_back_addr":"172.21.15.115:6807/28509","heartbeat_front_addr":"172.21.15.115:6805/28509","state":["exists","up"]},{"osd":6,"uuid":"c7d180be-21c5-4674-a9f2-10a1c7794871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":28731},{"type":"v1","addr":"172.21.15.152:6801","nonce":28731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":28731},{"type":"v1","addr":"172.21.15.152:6803","nonce":28731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":28731},{"type":"v1","addr":"172.21.15.152:6807","nonce":28731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":28731},{"type":"v1","addr":"172.21.15.152:6805","nonce":28731}]},"public_addr":"172.21.15.152:6801/28731","cluster_addr":"172.21.15.152:6803/28731","heartbeat_back_addr":"172.21.15.152:6807/28731","heartbeat_front_addr":"172.21.15.152:6805/28731","state":["exists","up"]},{"osd":7,"uuid":"4d2429ea-772f-4fb9-a421-240f1399f16c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":28836},{"type":"v1","addr":"172.21.15.175:6811","nonce":28836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":28836},{"type":"v1","addr":"172.21.15.175:6813","nonce":28836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":28836},{"type":"v1","addr":"172.21.15.175:6817","nonce":28836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":28836},{"type":"v1","addr":"172.21.15.175:6815","nonce":28836}]},"public_addr":"172.21.15.175:6811/28836","cluster_addr":"172.21.15.175:6813/28836","heartbeat_back_addr":"172.21.15.175:6817/28836","heartbeat_front_addr":"172.21.15.175:6815/28836","state":["exists","up"]},{"osd":8,"uuid":"de7389ab-a29e-4177-9e94-484fc6a94102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":28862},{"type":"v1","addr":"172.21.15.42:6817","nonce":28862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":28862},{"type":"v1","addr":"172.21.15.42:6819","nonce":28862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":28862},{"type":"v1","addr":"172.21.15.42:6823","nonce":28862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":28862},{"type":"v1","addr":"172.21.15.42:6821","nonce":28862}]},"public_addr":"172.21.15.42:6817/28862","cluster_addr":"172.21.15.42:6819/28862","heartbeat_back_addr":"172.21.15.42:6823/28862","heartbeat_front_addr":"172.21.15.42:6821/28862","state":["exists","up"]},{"osd":9,"uuid":"4b4b44da-97d6-41f2-8fc6-57680b10aeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":28510},{"type":"v1","addr":"172.21.15.115:6814","nonce":28510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":28510},{"type":"v1","addr":"172.21.15.115:6818","nonce":28510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":28510},{"type":"v1","addr":"172.21.15.115:6823","nonce":28510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":28510},{"type":"v1","addr":"172.21.15.115:6821","nonce":28510}]},"public_addr":"172.21.15.115:6814/28510","cluster_addr":"172.21.15.115:6818/28510","heartbeat_back_addr":"172.21.15.115:6823/28510","heartbeat_front_addr":"172.21.15.115:6821/28510","state":["exists","up"]},{"osd":10,"uuid":"5f285a07-9057-4575-b911-b3977acd397c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":28733},{"type":"v1","addr":"172.21.15.152:6817","nonce":28733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":28733},{"type":"v1","addr":"172.21.15.152:6819","nonce":28733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":28733},{"type":"v1","addr":"172.21.15.152:6824","nonce":28733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":28733},{"type":"v1","addr":"172.21.15.152:6821","nonce":28733}]},"public_addr":"172.21.15.152:6817/28733","cluster_addr":"172.21.15.152:6819/28733","heartbeat_back_addr":"172.21.15.152:6824/28733","heartbeat_front_addr":"172.21.15.152:6821/28733","state":["exists","up"]},{"osd":11,"uuid":"dacc0f3f-a598-4055-92f4-d41102ceca83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":28837},{"type":"v1","addr":"172.21.15.175:6827","nonce":28837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":28837},{"type":"v1","addr":"172.21.15.175:6829","nonce":28837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":28837},{"type":"v1","addr":"172.21.15.175:6833","nonce":28837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":28837},{"type":"v1","addr":"172.21.15.175:6831","nonce":28837}]},"public_addr":"172.21.15.175:6827/28837","cluster_addr":"172.21.15.175:6829/28837","heartbeat_back_addr":"172.21.15.175:6833/28837","heartbeat_front_addr":"172.21.15.175:6831/28837","state":["exists","up"]},{"osd":12,"uuid":"d218197b-d0db-410c-abff-8b849a21a555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":28866},{"type":"v1","addr":"172.21.15.42:6825","nonce":28866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":28866},{"type":"v1","addr":"172.21.15.42:6827","nonce":28866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":28866},{"type":"v1","addr":"172.21.15.42:6831","nonce":28866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":28866},{"type":"v1","addr":"172.21.15.42:6829","nonce":28866}]},"public_addr":"172.21.15.42:6825/28866","cluster_addr":"172.21.15.42:6827/28866","heartbeat_back_addr":"172.21.15.42:6831/28866","heartbeat_front_addr":"172.21.15.42:6829/28866","state":["exists","up"]},{"osd":13,"uuid":"c9bf5faf-f2ee-482e-b903-28c7718e2063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":28511},{"type":"v1","addr":"172.21.15.115:6825","nonce":28511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":28511},{"type":"v1","addr":"172.21.15.115:6827","nonce":28511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":28511},{"type":"v1","addr":"172.21.15.115:6831","nonce":28511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":28511},{"type":"v1","addr":"172.21.15.115:6829","nonce":28511}]},"public_addr":"172.21.15.115:6825/28511","cluster_addr":"172.21.15.115:6827/28511","heartbeat_back_addr":"172.21.15.115:6831/28511","heartbeat_front_addr":"172.21.15.115:6829/28511","state":["exists","up"]},{"osd":14,"uuid":"cc3f19cf-5a97-440b-a953-7727e6ed6a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6823","nonce":28734},{"type":"v1","addr":"172.21.15.152:6825","nonce":28734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":28734},{"type":"v1","addr":"172.21.15.152:6827","nonce":28734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":28734},{"type":"v1","addr":"172.21.15.152:6831","nonce":28734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":28734},{"type":"v1","addr":"172.21.15.152:6829","nonce":28734}]},"public_addr":"172.21.15.152:6825/28734","cluster_addr":"172.21.15.152:6827/28734","heartbeat_back_addr":"172.21.15.152:6831/28734","heartbeat_front_addr":"172.21.15.152:6829/28734","state":["exists","up"]},{"osd":15,"uuid":"51290c23-e86d-4e05-9ac8-ca53d6a1eb80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":28838},{"type":"v1","addr":"172.21.15.175:6819","nonce":28838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":28838},{"type":"v1","addr":"172.21.15.175:6821","nonce":28838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":28838},{"type":"v1","addr":"172.21.15.175:6825","nonce":28838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":28838},{"type":"v1","addr":"172.21.15.175:6823","nonce":28838}]},"public_addr":"172.21.15.175:6819/28838","cluster_addr":"172.21.15.175:6821/28838","heartbeat_back_addr":"172.21.15.175:6825/28838","heartbeat_front_addr":"172.21.15.175:6823/28838","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.539908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.544884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.614659+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.605046+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.503545+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.584788+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.563649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.622071+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.572859+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.526526+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.534845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.556801+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.565526+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.582172+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.619730+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:32:12.619890+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:32:26.083 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T01:32:26.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T01:32:26.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T01:32:26.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T01:32:26.085 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T01:32:26.085 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T01:32:26.086 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T01:32:26.086 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T01:32:26.086 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-15T01:32:26.087 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-15T01:32:26.087 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-15T01:32:26.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-15T01:32:26.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-09-15T01:32:26.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-09-15T01:32:26.089 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-09-15T01:32:26.089 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-09-15T01:32:26.492 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:26.492 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T01:32:26.506 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:26.506 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-15T01:32:26.508 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:26.509 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T01:32:26.517 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:26.517 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T01:32:26.537 INFO:teuthology.orchestra.run.smithi042.stdout:90194313222 2024-09-15T01:32:26.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T01:32:26.542 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:26.542 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-15T01:32:26.543 INFO:teuthology.orchestra.run.smithi042.stdout:90194313222 2024-09-15T01:32:26.543 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-15T01:32:26.551 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:26.552 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:26.553 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-15T01:32:26.553 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-15T01:32:26.554 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:26.554 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T01:32:26.568 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:26.568 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T01:32:26.580 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:26.581 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-15T01:32:26.589 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:26.589 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-15T01:32:26.616 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:26.616 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-15T01:32:26.620 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:26.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T01:32:26.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:32:26.657+0000 7f64fb7fe640 0 --2- 172.21.15.42:0/2299225437 >> v2:172.21.15.152:3300/0 conn(0x7f64fc06e590 0x7f64fc06a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T01:32:26.849 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:26.849 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T01:32:27.495 INFO:teuthology.orchestra.run.smithi042.stdout:90194313222 2024-09-15T01:32:27.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313222 got 90194313222 for osd.14 2024-09-15T01:32:27.529 INFO:teuthology.orchestra.run.smithi042.stdout:90194313222 2024-09-15T01:32:27.529 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.535 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:27.545 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:27.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313222 got 90194313222 for osd.0 2024-09-15T01:32:27.553 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.564 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:27.571 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.3 2024-09-15T01:32:27.571 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:27.577 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.5 2024-09-15T01:32:27.577 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.585 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.1 2024-09-15T01:32:27.585 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.12 2024-09-15T01:32:27.593 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.638 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:27.645 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:27.649 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:27.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.13 2024-09-15T01:32:27.663 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.8 2024-09-15T01:32:27.681 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.683 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:27.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.11 2024-09-15T01:32:27.685 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.701 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:27.703 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.10 2024-09-15T01:32:27.703 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.706 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:27.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.15 2024-09-15T01:32:27.716 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.4 2024-09-15T01:32:27.720 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.755 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:27.757 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:27.763 INFO:teuthology.orchestra.run.smithi042.stdout:90194313220 2024-09-15T01:32:27.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.6 2024-09-15T01:32:27.771 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.9 2024-09-15T01:32:27.771 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.2 2024-09-15T01:32:27.781 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.787 INFO:teuthology.orchestra.run.smithi042.stdout:90194313221 2024-09-15T01:32:27.800 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.7 2024-09-15T01:32:27.800 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:27.800 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T01:32:27.800 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:32:28.118 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:28.118 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:32:28.129 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-15T01:32:27.069479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":16,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":12737984,"kb_used_data":1486136940,"kb_used_omap":873,"kb_used_meta":0,"kb_avail":1486135872,"statfs":{"total":1534846828544,"available":1521803132928,"internally_reserved":1093632,"allocated":1521804226560,"data_stored":1521804226560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":894740,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.938338"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.582548+0000","last_change":"2024-09-15T01:32:25.582726+0000","last_active":"2024-09-15T01:32:25.582548+0000","last_peered":"2024-09-15T01:32:25.582548+0000","last_clean":"2024-09-15T01:32:25.582548+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:32:25.582548+0000","last_undegraded":"2024-09-15T01:32:25.582548+0000","last_fullsized":"2024-09-15T01:32:25.582548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.907962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.568349+0000","last_change":"2024-09-15T01:32:25.568523+0000","last_active":"2024-09-15T01:32:25.568349+0000","last_peered":"2024-09-15T01:32:25.568349+0000","last_clean":"2024-09-15T01:32:25.568349+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:32:25.568349+0000","last_undegraded":"2024-09-15T01:32:25.568349+0000","last_fullsized":"2024-09-15T01:32:25.568349+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.407227+0000","last_change":"2024-09-15T01:32:25.407396+0000","last_active":"2024-09-15T01:32:25.407227+0000","last_peered":"2024-09-15T01:32:25.407227+0000","last_clean":"2024-09-15T01:32:25.407227+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:32:25.407227+0000","last_undegraded":"2024-09-15T01:32:25.407227+0000","last_fullsized":"2024-09-15T01:32:25.407227+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:39.303667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.568036+0000","last_change":"2024-09-15T01:32:25.568260+0000","last_active":"2024-09-15T01:32:25.568036+0000","last_peered":"2024-09-15T01:32:25.568036+0000","last_clean":"2024-09-15T01:32:25.568036+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:32:25.568036+0000","last_undegraded":"2024-09-15T01:32:25.568036+0000","last_fullsized":"2024-09-15T01:32:25.568036+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:32.165806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.144113+0000","last_change":"2024-09-15T01:32:25.144113+0000","last_active":"2024-09-15T01:32:25.144113+0000","last_peered":"2024-09-15T01:32:25.144113+0000","last_clean":"2024-09-15T01:32:25.144113+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:32:25.144113+0000","last_undegraded":"2024-09-15T01:32:25.144113+0000","last_fullsized":"2024-09-15T01:32:25.144113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.583670+0000","last_change":"2024-09-15T01:32:25.583876+0000","last_active":"2024-09-15T01:32:25.583670+0000","last_peered":"2024-09-15T01:32:25.583670+0000","last_clean":"2024-09-15T01:32:25.583670+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:32:25.583670+0000","last_undegraded":"2024-09-15T01:32:25.583670+0000","last_fullsized":"2024-09-15T01:32:25.583670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:49.092006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.143524+0000","last_change":"2024-09-15T01:32:25.143709+0000","last_active":"2024-09-15T01:32:25.143524+0000","last_peered":"2024-09-15T01:32:25.143524+0000","last_clean":"2024-09-15T01:32:25.143524+0000","last_became_active":"2024-09-15T01:32:23.134382+0000","last_became_peered":"2024-09-15T01:32:23.134382+0000","last_unstale":"2024-09-15T01:32:25.143524+0000","last_undegraded":"2024-09-15T01:32:25.143524+0000","last_fullsized":"2024-09-15T01:32:25.143524+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.326224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.147778+0000","last_change":"2024-09-15T01:32:25.147954+0000","last_active":"2024-09-15T01:32:25.147778+0000","last_peered":"2024-09-15T01:32:25.147778+0000","last_clean":"2024-09-15T01:32:25.147778+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:32:25.147778+0000","last_undegraded":"2024-09-15T01:32:25.147778+0000","last_fullsized":"2024-09-15T01:32:25.147778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.228776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"24'112","reported_seq":149,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T01:32:24.467394+0000","last_change":"2024-09-15T01:32:16.749959+0000","last_active":"2024-09-15T01:32:24.467394+0000","last_peered":"2024-09-15T01:32:24.467394+0000","last_clean":"2024-09-15T01:32:24.467394+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:32:24.467394+0000","last_undegraded":"2024-09-15T01:32:24.467394+0000","last_fullsized":"2024-09-15T01:32:24.467394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_clean_scrub_stamp":"2024-09-15T01:32:15.451214+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795804,"kb_used_data":92883848,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883812,"statfs":{"total":95927926784,"available":95113023488,"internally_reserved":36864,"allocated":95113060352,"data_stored":95113060352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39907,"internal_metadata":0},"hb_peers":[0,1,2,3,4,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":9,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795896,"kb_used_data":92883768,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883720,"statfs":{"total":95927926784,"available":95112929280,"internally_reserved":49152,"allocated":95112978432,"data_stored":95112978432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39900,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796104,"kb_used_data":92883616,"kb_used_omap":55,"kb_used_meta":0,"kb_avail":92883512,"statfs":{"total":95927926784,"available":95112716288,"internally_reserved":106496,"allocated":95112822784,"data_stored":95112822784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56464,"internal_metadata":0},"hb_peers":[0,2,4,6,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":0,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796044,"kb_used_data":92883640,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":92883572,"statfs":{"total":95927926784,"available":95112777728,"internally_reserved":69632,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47148,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":814.68299999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]}]},{"osd":13,"up_from":21,"seq":90194313221,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":797260,"kb_used_data":92882448,"kb_used_omap":118,"kb_used_meta":0,"kb_avail":92882356,"statfs":{"total":95927926784,"available":95111532544,"internally_reserved":94208,"allocated":95111626752,"data_stored":95111626752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":121380,"internal_metadata":0},"hb_peers":[1,2,3,4,5,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]}]},{"osd":2,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":797216,"kb_used_data":92882448,"kb_used_omap":106,"kb_used_meta":0,"kb_avail":92882400,"statfs":{"total":95927926784,"available":95111577600,"internally_reserved":49152,"allocated":95111626752,"data_stored":95111626752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":109004,"internal_metadata":0},"hb_peers":[1,3,4,5,6,7,8,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":15,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796048,"kb_used_data":92883664,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92883568,"statfs":{"total":95927926784,"available":95112773632,"internally_reserved":98304,"allocated":95112871936,"data_stored":95112871936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":55134,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.603}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796068,"kb_used_data":92883640,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92883548,"statfs":{"total":95927926784,"available":95112753152,"internally_reserved":94208,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":54440,"internal_metadata":0},"hb_peers":[0,3,5,6,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":5,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796068,"kb_used_data":92883640,"kb_used_omap":52,"kb_used_meta":0,"kb_avail":92883548,"statfs":{"total":95927926784,"available":95112753152,"internally_reserved":94208,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53684,"internal_metadata":0},"hb_peers":[4,6,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]}]},{"osd":6,"up_from":21,"seq":90194313220,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796088,"kb_used_data":92883640,"kb_used_omap":58,"kb_used_meta":0,"kb_avail":92883528,"statfs":{"total":95927926784,"available":95112732672,"internally_reserved":114688,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":60292,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":7,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795668,"kb_used_data":92883948,"kb_used_omap":39,"kb_used_meta":0,"kb_avail":92883948,"statfs":{"total":95927926784,"available":95113162752,"internally_reserved":0,"allocated":95113162752,"data_stored":95113162752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39950,"internal_metadata":0},"hb_peers":[0,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":8,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796048,"kb_used_data":92883640,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":92883568,"statfs":{"total":95927926784,"available":95112773632,"internally_reserved":73728,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47192,"internal_metadata":0},"hb_peers":[0,6,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":10,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795896,"kb_used_data":92883768,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883720,"statfs":{"total":95927926784,"available":95112929280,"internally_reserved":49152,"allocated":95112978432,"data_stored":95112978432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39907,"internal_metadata":0},"hb_peers":[0,1,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":11,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795828,"kb_used_data":92883824,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883788,"statfs":{"total":95927926784,"available":95112998912,"internally_reserved":36864,"allocated":95113035776,"data_stored":95113035776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39907,"internal_metadata":0},"hb_peers":[0,1,2,3,4,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":12,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796052,"kb_used_data":92883640,"kb_used_omap":48,"kb_used_meta":0,"kb_avail":92883564,"statfs":{"total":95927926784,"available":95112769536,"internally_reserved":77824,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49833,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]}]},{"osd":1,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795896,"kb_used_data":92883768,"kb_used_omap":39,"kb_used_meta":0,"kb_avail":92883720,"statfs":{"total":95927926784,"available":95112929280,"internally_reserved":49152,"allocated":95112978432,"data_stored":95112978432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40598,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]}],"pool_statfs":[]}} 2024-09-15T01:32:28.131 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:32:28.411 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:28.412 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:32:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-15T01:32:27.069479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":16,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":12737984,"kb_used_data":1486136940,"kb_used_omap":873,"kb_used_meta":0,"kb_avail":1486135872,"statfs":{"total":1534846828544,"available":1521803132928,"internally_reserved":1093632,"allocated":1521804226560,"data_stored":1521804226560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":894740,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.938338"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.582548+0000","last_change":"2024-09-15T01:32:25.582726+0000","last_active":"2024-09-15T01:32:25.582548+0000","last_peered":"2024-09-15T01:32:25.582548+0000","last_clean":"2024-09-15T01:32:25.582548+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:32:25.582548+0000","last_undegraded":"2024-09-15T01:32:25.582548+0000","last_fullsized":"2024-09-15T01:32:25.582548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.907962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.568349+0000","last_change":"2024-09-15T01:32:25.568523+0000","last_active":"2024-09-15T01:32:25.568349+0000","last_peered":"2024-09-15T01:32:25.568349+0000","last_clean":"2024-09-15T01:32:25.568349+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:32:25.568349+0000","last_undegraded":"2024-09-15T01:32:25.568349+0000","last_fullsized":"2024-09-15T01:32:25.568349+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.407227+0000","last_change":"2024-09-15T01:32:25.407396+0000","last_active":"2024-09-15T01:32:25.407227+0000","last_peered":"2024-09-15T01:32:25.407227+0000","last_clean":"2024-09-15T01:32:25.407227+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:32:25.407227+0000","last_undegraded":"2024-09-15T01:32:25.407227+0000","last_fullsized":"2024-09-15T01:32:25.407227+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:39.303667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.568036+0000","last_change":"2024-09-15T01:32:25.568260+0000","last_active":"2024-09-15T01:32:25.568036+0000","last_peered":"2024-09-15T01:32:25.568036+0000","last_clean":"2024-09-15T01:32:25.568036+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:32:25.568036+0000","last_undegraded":"2024-09-15T01:32:25.568036+0000","last_fullsized":"2024-09-15T01:32:25.568036+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:32.165806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.144113+0000","last_change":"2024-09-15T01:32:25.144113+0000","last_active":"2024-09-15T01:32:25.144113+0000","last_peered":"2024-09-15T01:32:25.144113+0000","last_clean":"2024-09-15T01:32:25.144113+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:32:25.144113+0000","last_undegraded":"2024-09-15T01:32:25.144113+0000","last_fullsized":"2024-09-15T01:32:25.144113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.583670+0000","last_change":"2024-09-15T01:32:25.583876+0000","last_active":"2024-09-15T01:32:25.583670+0000","last_peered":"2024-09-15T01:32:25.583670+0000","last_clean":"2024-09-15T01:32:25.583670+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:32:25.583670+0000","last_undegraded":"2024-09-15T01:32:25.583670+0000","last_fullsized":"2024-09-15T01:32:25.583670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:49.092006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.143524+0000","last_change":"2024-09-15T01:32:25.143709+0000","last_active":"2024-09-15T01:32:25.143524+0000","last_peered":"2024-09-15T01:32:25.143524+0000","last_clean":"2024-09-15T01:32:25.143524+0000","last_became_active":"2024-09-15T01:32:23.134382+0000","last_became_peered":"2024-09-15T01:32:23.134382+0000","last_unstale":"2024-09-15T01:32:25.143524+0000","last_undegraded":"2024-09-15T01:32:25.143524+0000","last_fullsized":"2024-09-15T01:32:25.143524+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.326224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.147778+0000","last_change":"2024-09-15T01:32:25.147954+0000","last_active":"2024-09-15T01:32:25.147778+0000","last_peered":"2024-09-15T01:32:25.147778+0000","last_clean":"2024-09-15T01:32:25.147778+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:32:25.147778+0000","last_undegraded":"2024-09-15T01:32:25.147778+0000","last_fullsized":"2024-09-15T01:32:25.147778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.228776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"24'112","reported_seq":149,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T01:32:24.467394+0000","last_change":"2024-09-15T01:32:16.749959+0000","last_active":"2024-09-15T01:32:24.467394+0000","last_peered":"2024-09-15T01:32:24.467394+0000","last_clean":"2024-09-15T01:32:24.467394+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:32:24.467394+0000","last_undegraded":"2024-09-15T01:32:24.467394+0000","last_fullsized":"2024-09-15T01:32:24.467394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_clean_scrub_stamp":"2024-09-15T01:32:15.451214+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795804,"kb_used_data":92883848,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883812,"statfs":{"total":95927926784,"available":95113023488,"internally_reserved":36864,"allocated":95113060352,"data_stored":95113060352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39907,"internal_metadata":0},"hb_peers":[0,1,2,3,4,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":9,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795896,"kb_used_data":92883768,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883720,"statfs":{"total":95927926784,"available":95112929280,"internally_reserved":49152,"allocated":95112978432,"data_stored":95112978432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39900,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796104,"kb_used_data":92883616,"kb_used_omap":55,"kb_used_meta":0,"kb_avail":92883512,"statfs":{"total":95927926784,"available":95112716288,"internally_reserved":106496,"allocated":95112822784,"data_stored":95112822784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56464,"internal_metadata":0},"hb_peers":[0,2,4,6,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":0,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796044,"kb_used_data":92883640,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":92883572,"statfs":{"total":95927926784,"available":95112777728,"internally_reserved":69632,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47148,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":814.68299999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]}]},{"osd":13,"up_from":21,"seq":90194313221,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":797260,"kb_used_data":92882448,"kb_used_omap":118,"kb_used_meta":0,"kb_avail":92882356,"statfs":{"total":95927926784,"available":95111532544,"internally_reserved":94208,"allocated":95111626752,"data_stored":95111626752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":121380,"internal_metadata":0},"hb_peers":[1,2,3,4,5,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]}]},{"osd":2,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":797216,"kb_used_data":92882448,"kb_used_omap":106,"kb_used_meta":0,"kb_avail":92882400,"statfs":{"total":95927926784,"available":95111577600,"internally_reserved":49152,"allocated":95111626752,"data_stored":95111626752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":109004,"internal_metadata":0},"hb_peers":[1,3,4,5,6,7,8,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":15,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796048,"kb_used_data":92883664,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92883568,"statfs":{"total":95927926784,"available":95112773632,"internally_reserved":98304,"allocated":95112871936,"data_stored":95112871936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":55134,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.603}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796068,"kb_used_data":92883640,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92883548,"statfs":{"total":95927926784,"available":95112753152,"internally_reserved":94208,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":54440,"internal_metadata":0},"hb_peers":[0,3,5,6,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":5,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796068,"kb_used_data":92883640,"kb_used_omap":52,"kb_used_meta":0,"kb_avail":92883548,"statfs":{"total":95927926784,"available":95112753152,"internally_reserved":94208,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53684,"internal_metadata":0},"hb_peers":[4,6,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]}]},{"osd":6,"up_from":21,"seq":90194313220,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796088,"kb_used_data":92883640,"kb_used_omap":58,"kb_used_meta":0,"kb_avail":92883528,"statfs":{"total":95927926784,"available":95112732672,"internally_reserved":114688,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":60292,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":7,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795668,"kb_used_data":92883948,"kb_used_omap":39,"kb_used_meta":0,"kb_avail":92883948,"statfs":{"total":95927926784,"available":95113162752,"internally_reserved":0,"allocated":95113162752,"data_stored":95113162752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39950,"internal_metadata":0},"hb_peers":[0,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":8,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796048,"kb_used_data":92883640,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":92883568,"statfs":{"total":95927926784,"available":95112773632,"internally_reserved":73728,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47192,"internal_metadata":0},"hb_peers":[0,6,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":10,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795896,"kb_used_data":92883768,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883720,"statfs":{"total":95927926784,"available":95112929280,"internally_reserved":49152,"allocated":95112978432,"data_stored":95112978432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39907,"internal_metadata":0},"hb_peers":[0,1,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":11,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795828,"kb_used_data":92883824,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883788,"statfs":{"total":95927926784,"available":95112998912,"internally_reserved":36864,"allocated":95113035776,"data_stored":95113035776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39907,"internal_metadata":0},"hb_peers":[0,1,2,3,4,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":12,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796052,"kb_used_data":92883640,"kb_used_omap":48,"kb_used_meta":0,"kb_avail":92883564,"statfs":{"total":95927926784,"available":95112769536,"internally_reserved":77824,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49833,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]}]},{"osd":1,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795896,"kb_used_data":92883768,"kb_used_omap":39,"kb_used_meta":0,"kb_avail":92883720,"statfs":{"total":95927926784,"available":95112929280,"internally_reserved":49152,"allocated":95112978432,"data_stored":95112978432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40598,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]}],"pool_statfs":[]}} 2024-09-15T01:32:28.426 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T01:32:28.426 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T01:32:28.426 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T01:32:28.426 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T01:32:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:28.735 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T01:32:28.747 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T01:32:28.748 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-15T01:32:28.757 INFO:tasks.thrashosds:config is {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-15T01:32:28.757 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd sloppy crc': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'osd objectstore': 'filestore', 'osd sloppy crc': True}}, 'filestore': True, 'fs': 'xfs'}, 'install': {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-15T01:32:28.758 INFO:tasks.thrashosds:config is {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-15T01:32:28.758 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-15T01:32:29.055 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-15T01:32:29.055 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T01:32:29.055 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:32:29.334 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:29.334 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:32:29.349 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-15T01:32:29.070018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":16,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":12737984,"kb_used_data":1486136940,"kb_used_omap":873,"kb_used_meta":0,"kb_avail":1486135872,"statfs":{"total":1534846828544,"available":1521803132928,"internally_reserved":1093632,"allocated":1521804226560,"data_stored":1521804226560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":894740,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001576"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.582548+0000","last_change":"2024-09-15T01:32:25.582726+0000","last_active":"2024-09-15T01:32:25.582548+0000","last_peered":"2024-09-15T01:32:25.582548+0000","last_clean":"2024-09-15T01:32:25.582548+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:32:25.582548+0000","last_undegraded":"2024-09-15T01:32:25.582548+0000","last_fullsized":"2024-09-15T01:32:25.582548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.907962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.568349+0000","last_change":"2024-09-15T01:32:25.568523+0000","last_active":"2024-09-15T01:32:25.568349+0000","last_peered":"2024-09-15T01:32:25.568349+0000","last_clean":"2024-09-15T01:32:25.568349+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:32:25.568349+0000","last_undegraded":"2024-09-15T01:32:25.568349+0000","last_fullsized":"2024-09-15T01:32:25.568349+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.407227+0000","last_change":"2024-09-15T01:32:25.407396+0000","last_active":"2024-09-15T01:32:25.407227+0000","last_peered":"2024-09-15T01:32:25.407227+0000","last_clean":"2024-09-15T01:32:25.407227+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:32:25.407227+0000","last_undegraded":"2024-09-15T01:32:25.407227+0000","last_fullsized":"2024-09-15T01:32:25.407227+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:39.303667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.568036+0000","last_change":"2024-09-15T01:32:25.568260+0000","last_active":"2024-09-15T01:32:25.568036+0000","last_peered":"2024-09-15T01:32:25.568036+0000","last_clean":"2024-09-15T01:32:25.568036+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:32:25.568036+0000","last_undegraded":"2024-09-15T01:32:25.568036+0000","last_fullsized":"2024-09-15T01:32:25.568036+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:32.165806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.144113+0000","last_change":"2024-09-15T01:32:25.144113+0000","last_active":"2024-09-15T01:32:25.144113+0000","last_peered":"2024-09-15T01:32:25.144113+0000","last_clean":"2024-09-15T01:32:25.144113+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:32:25.144113+0000","last_undegraded":"2024-09-15T01:32:25.144113+0000","last_fullsized":"2024-09-15T01:32:25.144113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.583670+0000","last_change":"2024-09-15T01:32:25.583876+0000","last_active":"2024-09-15T01:32:25.583670+0000","last_peered":"2024-09-15T01:32:25.583670+0000","last_clean":"2024-09-15T01:32:25.583670+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:32:25.583670+0000","last_undegraded":"2024-09-15T01:32:25.583670+0000","last_fullsized":"2024-09-15T01:32:25.583670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:49.092006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.143524+0000","last_change":"2024-09-15T01:32:25.143709+0000","last_active":"2024-09-15T01:32:25.143524+0000","last_peered":"2024-09-15T01:32:25.143524+0000","last_clean":"2024-09-15T01:32:25.143524+0000","last_became_active":"2024-09-15T01:32:23.134382+0000","last_became_peered":"2024-09-15T01:32:23.134382+0000","last_unstale":"2024-09-15T01:32:25.143524+0000","last_undegraded":"2024-09-15T01:32:25.143524+0000","last_fullsized":"2024-09-15T01:32:25.143524+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.326224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.147778+0000","last_change":"2024-09-15T01:32:25.147954+0000","last_active":"2024-09-15T01:32:25.147778+0000","last_peered":"2024-09-15T01:32:25.147778+0000","last_clean":"2024-09-15T01:32:25.147778+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:32:25.147778+0000","last_undegraded":"2024-09-15T01:32:25.147778+0000","last_fullsized":"2024-09-15T01:32:25.147778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.228776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"24'112","reported_seq":149,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T01:32:24.467394+0000","last_change":"2024-09-15T01:32:16.749959+0000","last_active":"2024-09-15T01:32:24.467394+0000","last_peered":"2024-09-15T01:32:24.467394+0000","last_clean":"2024-09-15T01:32:24.467394+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:32:24.467394+0000","last_undegraded":"2024-09-15T01:32:24.467394+0000","last_fullsized":"2024-09-15T01:32:24.467394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_clean_scrub_stamp":"2024-09-15T01:32:15.451214+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795804,"kb_used_data":92883848,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883812,"statfs":{"total":95927926784,"available":95113023488,"internally_reserved":36864,"allocated":95113060352,"data_stored":95113060352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39907,"internal_metadata":0},"hb_peers":[0,1,2,3,4,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":9,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795896,"kb_used_data":92883768,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883720,"statfs":{"total":95927926784,"available":95112929280,"internally_reserved":49152,"allocated":95112978432,"data_stored":95112978432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39900,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796104,"kb_used_data":92883616,"kb_used_omap":55,"kb_used_meta":0,"kb_avail":92883512,"statfs":{"total":95927926784,"available":95112716288,"internally_reserved":106496,"allocated":95112822784,"data_stored":95112822784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56464,"internal_metadata":0},"hb_peers":[0,2,4,6,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":0,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796044,"kb_used_data":92883640,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":92883572,"statfs":{"total":95927926784,"available":95112777728,"internally_reserved":69632,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47148,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":814.68299999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]}]},{"osd":13,"up_from":21,"seq":90194313221,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":797260,"kb_used_data":92882448,"kb_used_omap":118,"kb_used_meta":0,"kb_avail":92882356,"statfs":{"total":95927926784,"available":95111532544,"internally_reserved":94208,"allocated":95111626752,"data_stored":95111626752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":121380,"internal_metadata":0},"hb_peers":[1,2,3,4,5,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]}]},{"osd":2,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":797216,"kb_used_data":92882448,"kb_used_omap":106,"kb_used_meta":0,"kb_avail":92882400,"statfs":{"total":95927926784,"available":95111577600,"internally_reserved":49152,"allocated":95111626752,"data_stored":95111626752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":109004,"internal_metadata":0},"hb_peers":[1,3,4,5,6,7,8,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":15,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796048,"kb_used_data":92883664,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92883568,"statfs":{"total":95927926784,"available":95112773632,"internally_reserved":98304,"allocated":95112871936,"data_stored":95112871936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":55134,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.603}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796068,"kb_used_data":92883640,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92883548,"statfs":{"total":95927926784,"available":95112753152,"internally_reserved":94208,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":54440,"internal_metadata":0},"hb_peers":[0,3,5,6,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":5,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796068,"kb_used_data":92883640,"kb_used_omap":52,"kb_used_meta":0,"kb_avail":92883548,"statfs":{"total":95927926784,"available":95112753152,"internally_reserved":94208,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53684,"internal_metadata":0},"hb_peers":[4,6,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]}]},{"osd":6,"up_from":21,"seq":90194313220,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796088,"kb_used_data":92883640,"kb_used_omap":58,"kb_used_meta":0,"kb_avail":92883528,"statfs":{"total":95927926784,"available":95112732672,"internally_reserved":114688,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":60292,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":7,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795668,"kb_used_data":92883948,"kb_used_omap":39,"kb_used_meta":0,"kb_avail":92883948,"statfs":{"total":95927926784,"available":95113162752,"internally_reserved":0,"allocated":95113162752,"data_stored":95113162752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39950,"internal_metadata":0},"hb_peers":[0,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":8,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796048,"kb_used_data":92883640,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":92883568,"statfs":{"total":95927926784,"available":95112773632,"internally_reserved":73728,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47192,"internal_metadata":0},"hb_peers":[0,6,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":10,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795896,"kb_used_data":92883768,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883720,"statfs":{"total":95927926784,"available":95112929280,"internally_reserved":49152,"allocated":95112978432,"data_stored":95112978432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39907,"internal_metadata":0},"hb_peers":[0,1,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":11,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795828,"kb_used_data":92883824,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883788,"statfs":{"total":95927926784,"available":95112998912,"internally_reserved":36864,"allocated":95113035776,"data_stored":95113035776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39907,"internal_metadata":0},"hb_peers":[0,1,2,3,4,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":12,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796052,"kb_used_data":92883640,"kb_used_omap":48,"kb_used_meta":0,"kb_avail":92883564,"statfs":{"total":95927926784,"available":95112769536,"internally_reserved":77824,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49833,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]}]},{"osd":1,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795896,"kb_used_data":92883768,"kb_used_omap":39,"kb_used_meta":0,"kb_avail":92883720,"statfs":{"total":95927926784,"available":95112929280,"internally_reserved":49152,"allocated":95112978432,"data_stored":95112978432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40598,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]}],"pool_statfs":[]}} 2024-09-15T01:32:29.351 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:32:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:29.628 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:32:29.640 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-15T01:32:29.070018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":16,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":12737984,"kb_used_data":1486136940,"kb_used_omap":873,"kb_used_meta":0,"kb_avail":1486135872,"statfs":{"total":1534846828544,"available":1521803132928,"internally_reserved":1093632,"allocated":1521804226560,"data_stored":1521804226560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":894740,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001576"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.582548+0000","last_change":"2024-09-15T01:32:25.582726+0000","last_active":"2024-09-15T01:32:25.582548+0000","last_peered":"2024-09-15T01:32:25.582548+0000","last_clean":"2024-09-15T01:32:25.582548+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:32:25.582548+0000","last_undegraded":"2024-09-15T01:32:25.582548+0000","last_fullsized":"2024-09-15T01:32:25.582548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.907962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.568349+0000","last_change":"2024-09-15T01:32:25.568523+0000","last_active":"2024-09-15T01:32:25.568349+0000","last_peered":"2024-09-15T01:32:25.568349+0000","last_clean":"2024-09-15T01:32:25.568349+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:32:25.568349+0000","last_undegraded":"2024-09-15T01:32:25.568349+0000","last_fullsized":"2024-09-15T01:32:25.568349+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.407227+0000","last_change":"2024-09-15T01:32:25.407396+0000","last_active":"2024-09-15T01:32:25.407227+0000","last_peered":"2024-09-15T01:32:25.407227+0000","last_clean":"2024-09-15T01:32:25.407227+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:32:25.407227+0000","last_undegraded":"2024-09-15T01:32:25.407227+0000","last_fullsized":"2024-09-15T01:32:25.407227+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:39.303667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.568036+0000","last_change":"2024-09-15T01:32:25.568260+0000","last_active":"2024-09-15T01:32:25.568036+0000","last_peered":"2024-09-15T01:32:25.568036+0000","last_clean":"2024-09-15T01:32:25.568036+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:32:25.568036+0000","last_undegraded":"2024-09-15T01:32:25.568036+0000","last_fullsized":"2024-09-15T01:32:25.568036+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:32.165806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.144113+0000","last_change":"2024-09-15T01:32:25.144113+0000","last_active":"2024-09-15T01:32:25.144113+0000","last_peered":"2024-09-15T01:32:25.144113+0000","last_clean":"2024-09-15T01:32:25.144113+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:32:25.144113+0000","last_undegraded":"2024-09-15T01:32:25.144113+0000","last_fullsized":"2024-09-15T01:32:25.144113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.583670+0000","last_change":"2024-09-15T01:32:25.583876+0000","last_active":"2024-09-15T01:32:25.583670+0000","last_peered":"2024-09-15T01:32:25.583670+0000","last_clean":"2024-09-15T01:32:25.583670+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:32:25.583670+0000","last_undegraded":"2024-09-15T01:32:25.583670+0000","last_fullsized":"2024-09-15T01:32:25.583670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:49.092006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.143524+0000","last_change":"2024-09-15T01:32:25.143709+0000","last_active":"2024-09-15T01:32:25.143524+0000","last_peered":"2024-09-15T01:32:25.143524+0000","last_clean":"2024-09-15T01:32:25.143524+0000","last_became_active":"2024-09-15T01:32:23.134382+0000","last_became_peered":"2024-09-15T01:32:23.134382+0000","last_unstale":"2024-09-15T01:32:25.143524+0000","last_undegraded":"2024-09-15T01:32:25.143524+0000","last_fullsized":"2024-09-15T01:32:25.143524+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.326224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-15T01:32:25.147778+0000","last_change":"2024-09-15T01:32:25.147954+0000","last_active":"2024-09-15T01:32:25.147778+0000","last_peered":"2024-09-15T01:32:25.147778+0000","last_clean":"2024-09-15T01:32:25.147778+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:32:25.147778+0000","last_undegraded":"2024-09-15T01:32:25.147778+0000","last_fullsized":"2024-09-15T01:32:25.147778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.228776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"24'112","reported_seq":149,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T01:32:24.467394+0000","last_change":"2024-09-15T01:32:16.749959+0000","last_active":"2024-09-15T01:32:24.467394+0000","last_peered":"2024-09-15T01:32:24.467394+0000","last_clean":"2024-09-15T01:32:24.467394+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:32:24.467394+0000","last_undegraded":"2024-09-15T01:32:24.467394+0000","last_fullsized":"2024-09-15T01:32:24.467394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_clean_scrub_stamp":"2024-09-15T01:32:15.451214+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795804,"kb_used_data":92883848,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883812,"statfs":{"total":95927926784,"available":95113023488,"internally_reserved":36864,"allocated":95113060352,"data_stored":95113060352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39907,"internal_metadata":0},"hb_peers":[0,1,2,3,4,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":9,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795896,"kb_used_data":92883768,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883720,"statfs":{"total":95927926784,"available":95112929280,"internally_reserved":49152,"allocated":95112978432,"data_stored":95112978432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39900,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796104,"kb_used_data":92883616,"kb_used_omap":55,"kb_used_meta":0,"kb_avail":92883512,"statfs":{"total":95927926784,"available":95112716288,"internally_reserved":106496,"allocated":95112822784,"data_stored":95112822784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56464,"internal_metadata":0},"hb_peers":[0,2,4,6,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":0,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796044,"kb_used_data":92883640,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":92883572,"statfs":{"total":95927926784,"available":95112777728,"internally_reserved":69632,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47148,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":814.68299999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]}]},{"osd":13,"up_from":21,"seq":90194313221,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":797260,"kb_used_data":92882448,"kb_used_omap":118,"kb_used_meta":0,"kb_avail":92882356,"statfs":{"total":95927926784,"available":95111532544,"internally_reserved":94208,"allocated":95111626752,"data_stored":95111626752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":121380,"internal_metadata":0},"hb_peers":[1,2,3,4,5,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]}]},{"osd":2,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":797216,"kb_used_data":92882448,"kb_used_omap":106,"kb_used_meta":0,"kb_avail":92882400,"statfs":{"total":95927926784,"available":95111577600,"internally_reserved":49152,"allocated":95111626752,"data_stored":95111626752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":109004,"internal_metadata":0},"hb_peers":[1,3,4,5,6,7,8,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":15,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796048,"kb_used_data":92883664,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92883568,"statfs":{"total":95927926784,"available":95112773632,"internally_reserved":98304,"allocated":95112871936,"data_stored":95112871936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":55134,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.603}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796068,"kb_used_data":92883640,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92883548,"statfs":{"total":95927926784,"available":95112753152,"internally_reserved":94208,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":54440,"internal_metadata":0},"hb_peers":[0,3,5,6,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":5,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796068,"kb_used_data":92883640,"kb_used_omap":52,"kb_used_meta":0,"kb_avail":92883548,"statfs":{"total":95927926784,"available":95112753152,"internally_reserved":94208,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53684,"internal_metadata":0},"hb_peers":[4,6,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]}]},{"osd":6,"up_from":21,"seq":90194313220,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796088,"kb_used_data":92883640,"kb_used_omap":58,"kb_used_meta":0,"kb_avail":92883528,"statfs":{"total":95927926784,"available":95112732672,"internally_reserved":114688,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":60292,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":7,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795668,"kb_used_data":92883948,"kb_used_omap":39,"kb_used_meta":0,"kb_avail":92883948,"statfs":{"total":95927926784,"available":95113162752,"internally_reserved":0,"allocated":95113162752,"data_stored":95113162752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39950,"internal_metadata":0},"hb_peers":[0,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":8,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796048,"kb_used_data":92883640,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":92883568,"statfs":{"total":95927926784,"available":95112773632,"internally_reserved":73728,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47192,"internal_metadata":0},"hb_peers":[0,6,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":10,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795896,"kb_used_data":92883768,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883720,"statfs":{"total":95927926784,"available":95112929280,"internally_reserved":49152,"allocated":95112978432,"data_stored":95112978432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39907,"internal_metadata":0},"hb_peers":[0,1,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":11,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795828,"kb_used_data":92883824,"kb_used_omap":38,"kb_used_meta":0,"kb_avail":92883788,"statfs":{"total":95927926784,"available":95112998912,"internally_reserved":36864,"allocated":95113035776,"data_stored":95113035776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39907,"internal_metadata":0},"hb_peers":[0,1,2,3,4,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":12,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796052,"kb_used_data":92883640,"kb_used_omap":48,"kb_used_meta":0,"kb_avail":92883564,"statfs":{"total":95927926784,"available":95112769536,"internally_reserved":77824,"allocated":95112847360,"data_stored":95112847360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49833,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]}]},{"osd":1,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795896,"kb_used_data":92883768,"kb_used_omap":39,"kb_used_meta":0,"kb_avail":92883720,"statfs":{"total":95927926784,"available":95112929280,"internally_reserved":49152,"allocated":95112978432,"data_stored":95112978432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40598,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]}],"pool_statfs":[]}} 2024-09-15T01:32:29.641 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T01:32:29.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-15T01:32:29.920 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 28 2024-09-15T01:32:29.920 INFO:teuthology.orchestra.run.smithi042.stdout:fsid d4803586-ed44-4154-8ee1-6b95ec3e17e7 2024-09-15T01:32:29.920 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-15T01:32:07.285177+0000 2024-09-15T01:32:29.920 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-15T01:32:25.135277+0000 2024-09-15T01:32:29.920 INFO:teuthology.orchestra.run.smithi042.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T01:32:29.920 INFO:teuthology.orchestra.run.smithi042.stdout:crush_version 4 2024-09-15T01:32:29.920 INFO:teuthology.orchestra.run.smithi042.stdout:full_ratio 0.95 2024-09-15T01:32:29.920 INFO:teuthology.orchestra.run.smithi042.stdout:backfillfull_ratio 0.9 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:nearfull_ratio 0.85 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:require_min_compat_client luminous 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:min_compat_client jewel 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:require_osd_release quincy 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:stretch_mode_enabled false 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 24 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 28 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: removed_snaps_queue [2~1] 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:max_osd 16 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6808/28861,v1:172.21.15.42:6809/28861] [v2:172.21.15.42:6810/28861,v1:172.21.15.42:6811/28861] exists,up bec41583-a0a4-4712-a9ec-eec1e88fb60f 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/28508,v1:172.21.15.115:6809/28508] [v2:172.21.15.115:6810/28508,v1:172.21.15.115:6811/28508] exists,up 2cca66be-8685-4a38-abf8-c864cd5434da 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6808/28732,v1:172.21.15.152:6809/28732] [v2:172.21.15.152:6810/28732,v1:172.21.15.152:6811/28732] exists,up b8756aac-3a3e-41ba-a6ef-497063cb44df 2024-09-15T01:32:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6802/28835,v1:172.21.15.175:6803/28835] [v2:172.21.15.175:6804/28835,v1:172.21.15.175:6805/28835] exists,up 554875f5-74c0-4b58-80ee-6ba93748f104 2024-09-15T01:32:29.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6800/28860,v1:172.21.15.42:6801/28860] [v2:172.21.15.42:6802/28860,v1:172.21.15.42:6803/28860] exists,up 3fe2c05a-4c22-4dd7-a8c6-282155c82eb8 2024-09-15T01:32:29.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/28509,v1:172.21.15.115:6801/28509] [v2:172.21.15.115:6802/28509,v1:172.21.15.115:6803/28509] exists,up a301d1ce-3e31-4782-ade3-f9c0f1e6a95d 2024-09-15T01:32:29.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6800/28731,v1:172.21.15.152:6801/28731] [v2:172.21.15.152:6802/28731,v1:172.21.15.152:6803/28731] exists,up c7d180be-21c5-4674-a9f2-10a1c7794871 2024-09-15T01:32:29.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6810/28836,v1:172.21.15.175:6811/28836] [v2:172.21.15.175:6812/28836,v1:172.21.15.175:6813/28836] exists,up 4d2429ea-772f-4fb9-a421-240f1399f16c 2024-09-15T01:32:29.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6816/28862,v1:172.21.15.42:6817/28862] [v2:172.21.15.42:6818/28862,v1:172.21.15.42:6819/28862] exists,up de7389ab-a29e-4177-9e94-484fc6a94102 2024-09-15T01:32:29.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6812/28510,v1:172.21.15.115:6814/28510] [v2:172.21.15.115:6816/28510,v1:172.21.15.115:6818/28510] exists,up 4b4b44da-97d6-41f2-8fc6-57680b10aeb8 2024-09-15T01:32:29.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6815/28733,v1:172.21.15.152:6817/28733] [v2:172.21.15.152:6818/28733,v1:172.21.15.152:6819/28733] exists,up 5f285a07-9057-4575-b911-b3977acd397c 2024-09-15T01:32:29.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6826/28837,v1:172.21.15.175:6827/28837] [v2:172.21.15.175:6828/28837,v1:172.21.15.175:6829/28837] exists,up dacc0f3f-a598-4055-92f4-d41102ceca83 2024-09-15T01:32:29.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6824/28866,v1:172.21.15.42:6825/28866] [v2:172.21.15.42:6826/28866,v1:172.21.15.42:6827/28866] exists,up d218197b-d0db-410c-abff-8b849a21a555 2024-09-15T01:32:29.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/28511,v1:172.21.15.115:6825/28511] [v2:172.21.15.115:6826/28511,v1:172.21.15.115:6827/28511] exists,up c9bf5faf-f2ee-482e-b903-28c7718e2063 2024-09-15T01:32:29.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] [v2:172.21.15.152:6826/28734,v1:172.21.15.152:6827/28734] exists,up cc3f19cf-5a97-440b-a953-7727e6ed6a0d 2024-09-15T01:32:29.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] [v2:172.21.15.175:6820/28838,v1:172.21.15.175:6821/28838] exists,up 51290c23-e86d-4e05-9ac8-ca53d6a1eb80 2024-09-15T01:32:29.932 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6808/28861,v1:172.21.15.42:6809/28861] [v2:172.21.15.42:6810/28861,v1:172.21.15.42:6811/28861] exists,up bec41583-a0a4-4712-a9ec-eec1e88fb60f', 'osd.1 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/28508,v1:172.21.15.115:6809/28508] [v2:172.21.15.115:6810/28508,v1:172.21.15.115:6811/28508] exists,up 2cca66be-8685-4a38-abf8-c864cd5434da', 'osd.2 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6808/28732,v1:172.21.15.152:6809/28732] [v2:172.21.15.152:6810/28732,v1:172.21.15.152:6811/28732] exists,up b8756aac-3a3e-41ba-a6ef-497063cb44df', 'osd.3 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6802/28835,v1:172.21.15.175:6803/28835] [v2:172.21.15.175:6804/28835,v1:172.21.15.175:6805/28835] exists,up 554875f5-74c0-4b58-80ee-6ba93748f104', 'osd.4 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6800/28860,v1:172.21.15.42:6801/28860] [v2:172.21.15.42:6802/28860,v1:172.21.15.42:6803/28860] exists,up 3fe2c05a-4c22-4dd7-a8c6-282155c82eb8', 'osd.5 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/28509,v1:172.21.15.115:6801/28509] [v2:172.21.15.115:6802/28509,v1:172.21.15.115:6803/28509] exists,up a301d1ce-3e31-4782-ade3-f9c0f1e6a95d', 'osd.6 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6800/28731,v1:172.21.15.152:6801/28731] [v2:172.21.15.152:6802/28731,v1:172.21.15.152:6803/28731] exists,up c7d180be-21c5-4674-a9f2-10a1c7794871', 'osd.7 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6810/28836,v1:172.21.15.175:6811/28836] [v2:172.21.15.175:6812/28836,v1:172.21.15.175:6813/28836] exists,up 4d2429ea-772f-4fb9-a421-240f1399f16c', 'osd.8 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6816/28862,v1:172.21.15.42:6817/28862] [v2:172.21.15.42:6818/28862,v1:172.21.15.42:6819/28862] exists,up de7389ab-a29e-4177-9e94-484fc6a94102', 'osd.9 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6812/28510,v1:172.21.15.115:6814/28510] [v2:172.21.15.115:6816/28510,v1:172.21.15.115:6818/28510] exists,up 4b4b44da-97d6-41f2-8fc6-57680b10aeb8', 'osd.10 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6815/28733,v1:172.21.15.152:6817/28733] [v2:172.21.15.152:6818/28733,v1:172.21.15.152:6819/28733] exists,up 5f285a07-9057-4575-b911-b3977acd397c', 'osd.11 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6826/28837,v1:172.21.15.175:6827/28837] [v2:172.21.15.175:6828/28837,v1:172.21.15.175:6829/28837] exists,up dacc0f3f-a598-4055-92f4-d41102ceca83', 'osd.12 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6824/28866,v1:172.21.15.42:6825/28866] [v2:172.21.15.42:6826/28866,v1:172.21.15.42:6827/28866] exists,up d218197b-d0db-410c-abff-8b849a21a555', 'osd.13 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/28511,v1:172.21.15.115:6825/28511] [v2:172.21.15.115:6826/28511,v1:172.21.15.115:6827/28511] exists,up c9bf5faf-f2ee-482e-b903-28c7718e2063', 'osd.14 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] [v2:172.21.15.152:6826/28734,v1:172.21.15.152:6827/28734] exists,up cc3f19cf-5a97-440b-a953-7727e6ed6a0d', 'osd.15 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] [v2:172.21.15.175:6820/28838,v1:172.21.15.175:6821/28838] exists,up 51290c23-e86d-4e05-9ac8-ca53d6a1eb80'] 2024-09-15T01:32:29.933 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-15T01:32:30.030 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:30.030 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mon.a", 2024-09-15T01:32:30.030 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster": "ceph", 2024-09-15T01:32:30.030 INFO:teuthology.orchestra.run.smithi042.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-15T01:32:30.030 INFO:teuthology.orchestra.run.smithi042.stdout: "admin_socket_mode": "", 2024-09-15T01:32:30.030 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-15T01:32:30.030 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_client_required": "cephx, none", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_cluster_required": "cephx", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_debug": "true", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_service_required": "cephx", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_supported": "cephx", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio": "true", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio_poll_ms": "250", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio_reap_max": "16", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_async_discard": "false", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_block_size": "4096", 2024-09-15T01:32:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_aio": "false", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_inflight_ios": "false", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_enable_discard": "false", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_flock_retry": "3", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_inject_crash": "0", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_ioring": "false", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_ioring_hipri": "false", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-15T01:32:30.032 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_type": "", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_alloc_size": "1048576", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_allocator": "hybrid", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_buffered_io": "true", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_check_for_zeros": "false", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_compact_log_sync": "false", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-15T01:32:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_max_log_runway": "4194304", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_max_prefetch": "1048576", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_min_flush_size": "524288", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_min_log_runway": "1048576", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_replay_recovery": "false", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_sync_write": "false", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_allocation_from_file": "true", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_allocator": "hybrid", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_async_db_compaction": "true", 2024-09-15T01:32:30.034 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-15T01:32:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-15T01:32:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-15T01:32:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-15T01:32:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-15T01:32:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-15T01:32:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-15T01:32:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_create": "true", 2024-09-15T01:32:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_db_create": "false", 2024-09-15T01:32:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_db_path": "", 2024-09-15T01:32:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_db_size": "0", 2024-09-15T01:32:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_path": "", 2024-09-15T01:32:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_preallocate_file": "false", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_size": "107374182400", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_wal_create": "false", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_wal_path": "", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_wal_size": "100663296", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs": "true", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T01:32:30.036 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_autotune": "true", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_size": "0", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_type": "2q", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_clone_cow": "true", 2024-09-15T01:32:30.037 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_mode": "none", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_csum_type": "crc32c", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_freelist": "false", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-15T01:32:30.038 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_small_allocations": "0", 2024-09-15T01:32:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_default_buffered_read": "true", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_default_buffered_write": "false", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fail_eio": "false", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-15T01:32:30.040 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mount": "false", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_umount": "false", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-15T01:32:30.041 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_ignore_data_csum": "false", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_log_op_age": "5.000000", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_alloc_size": "0", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_blob_size": "0", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_deferred_txc": "32", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_min_alloc_size": "0", 2024-09-15T01:32:30.042 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_nid_prealloc": "1024", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_retry_disk_reads": "3", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_cf": "true", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-15T01:32:30.043 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_mem": "512", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_tracing": "false", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-15T01:32:30.044 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_zero_block_detection": "false", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-15T01:32:30.045 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_cluster_require_signatures": "false", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_cluster_require_version": "2", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_require_signatures": "false", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_require_version": "2", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_service_require_signatures": "false", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_service_require_version": "2", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_sign_messages": "true", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "chdir": "", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_acl_type": "", 2024-09-15T01:32:30.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_asio_thread_count": "2", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_cache_mid": "0.750000", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_cache_size": "16384", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_caps_release_delay": "5", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_check_pool_perm": "true", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_force_sync_read": "false", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_getattr_caps": "false", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_inject_features": "", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_inject_tick_delay": "0", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_die_on_failed_remount": "false", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_dirsize_rbytes": "true", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_force_lazyio": "false", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_fs": "", 2024-09-15T01:32:30.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_inject_release_failure": "false", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_max_inline_size": "4096", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mds_namespace": "", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_metadata": "", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mount_gid": "-1", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mount_timeout": "300", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mount_uid": "-1", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mountpoint": "/", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_notify_timeout": "10", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc": "true", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_max_dirty": "104857600", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_max_objects": "1000", 2024-09-15T01:32:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_size": "209715200", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_target_dirty": "8388608", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_permissions": "true", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_quota": "true", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_quota_df": "true", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_readahead_max_bytes": "0", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_readahead_max_periods": "4", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_readahead_min": "131072", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_reconnect_stale": "false", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_shutdown_timeout": "30", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_snapdir": ".snap", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_tick_interval": "1", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_trace": "", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_try_dentry_invalidate": "false", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_use_faked_inos": "false", 2024-09-15T01:32:30.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_use_random_mds": "false", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_graylog": "false", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_graylog_port": "12201", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_monitors": "default=true", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_syslog": "false", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_syslog_level": "info", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "-", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_network": "", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_network_interface": "", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zlib_isal": "false", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zlib_level": "5", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zlib_winsize": "-15", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zstd_level": "1", 2024-09-15T01:32:30.050 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_alien_op_num_threads": "6", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_location": "", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_location_hook": "", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_location_hook_timeout": "10", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "daemonize": "false", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_alienstore": "0/5", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_allow_any_pool_priority": "false", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_asok": "1/5", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_asok_assert_abort": "false", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_asserts_on_shutdown": "true", 2024-09-15T01:32:30.051 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_auth": "20/20", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_bdev": "1/3", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_bluefs": "1/5", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_bluestore": "1/5", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_buffer": "0/1", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_ceph_exporter": "1/5", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_cephfs_mirror": "0/5", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_cephsqlite": "0/5", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_civetweb": "1/10", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_client": "0/5", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_compressor": "1/5", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_context": "0/1", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_crush": "1/1", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_crypto": "1/5", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_deliberately_leak_memory": "false", 2024-09-15T01:32:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_disable_randomized_ping": "false", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_dpdk": "1/5", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_eventtrace": "1/5", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_filer": "0/1", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_filestore": "1/3", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_finisher": "1/1", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_fuse": "1/5", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_heartbeat_testing_span": "0", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_heartbeatmap": "1/5", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_javaclient": "1/5", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_journal": "1/3", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_journaler": "0/5", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_kstore": "1/5", 2024-09-15T01:32:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_leveldb": "4/5", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_lockdep": "0/1", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mclock": "1/5", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds": "1/5", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_balancer": "1/5", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_locker": "1/5", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_log": "1/5", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_log_expire": "1/5", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_migrator": "1/5", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_memdb": "4/5", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mgr": "2/5", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mgrc": "1/5", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mon": "20/20", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_monc": "0/10", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_ms": "1/1", 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_none": "0/5", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_objclass": "0/5", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_objectcacher": "0/5", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_objecter": "0/1", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_optracker": "0/5", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_osd": "1/5", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_paxos": "20/20", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_perfcounter": "1/5", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_prioritycache": "1/5", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rados": "0/5", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd": "0/5", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd_mirror": "0/5", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd_pwl": "0/5", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd_replay": "0/5", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_refs": "0/0", 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_reserver": "1/1", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw": "1/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_datacache": "1/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_sync": "1/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rocksdb": "4/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore": "0/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_cache": "0/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_cleaner": "0/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_device": "0/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_journal": "0/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_lba": "0/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_odata": "0/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_omap": "0/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_onode": "0/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_tm": "0/5", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_striper": "0/1", 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_test": "0/5", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_throttle": "1/1", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_timer": "0/1", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_tp": "0/5", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "device_failure_prediction_mode": "none", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_graylog": "false", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_journald": "false", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_stderr": "true", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_syslog": "false", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event_tracing": "false", 2024-09-15T01:32:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_addr": "0.0.0.0", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_http_port": "9926", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_prio_limit": "5", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_sort_metrics": "true", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_stats_period": "5", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "fake_statfs_for_testing": "0", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "fatal_signal_handlers": "true", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "filer_max_purge_ops": "10", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "filer_max_truncate_ops": "128", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_apply_finisher_threads": "1", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_blackhole": "false", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_btrfs_clone_range": "true", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_btrfs_snap": "true", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_caller_concurrency": "10", 2024-09-15T01:32:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_collect_device_partition_information": "true", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_commit_timeout": "600.000000", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_inject_read_err": "false", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_omap_check": "false", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_verify_split": "false", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_dump_file": "", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fadvise": "true", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fail_eio": "true", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fd_cache_shards": "16", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fd_cache_size": "128", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fiemap": "false", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fiemap_threshold": "4096", 2024-09-15T01:32:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_inject_stall": "0", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_journal_parallel": "false", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_journal_trailing": "false", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_journal_writeahead": "false", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_kill_at": "0", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs": "0", 2024-09-15T01:32:30.060 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size": "0", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_merge_threshold": "-10", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_odsync_write": "false", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_omap_backend": "rocksdb", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_omap_backend_path": "", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-15T01:32:30.061 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_op_thread_timeout": "60", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_op_threads": "2", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_punch_hole": "false", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_ops": "50", 2024-09-15T01:32:30.062 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_seek_data_hole": "false", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_sloppy_crc": "false", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_splice": "false", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_split_multiple": "2", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_split_rand_factor": "20", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_update_to": "1000", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-15T01:32:30.063 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_enable": "true", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_xfs_extsize": "false", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_zfs_snap": "false", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "fio_dir": "/tmp/fio", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "d4803586-ed44-4154-8ee1-6b95ec3e17e7", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_allow_other": "true", 2024-09-15T01:32:30.064 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_atomic_o_trunc": "true", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_big_writes": "true", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_debug": "false", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_default_permissions": "false", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_disable_pagecache": "false", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_max_write": "0", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_multithreaded": "true", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_require_active_mds": "true", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_set_user_groups": "true", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_splice_move": "true", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_splice_read": "true", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_splice_write": "true", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_use_invalidate_cb": "true", 2024-09-15T01:32:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "gss_target_name": "ceph", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_file": "", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_inject_failure": "0", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_interval": "5", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "host": "smithi042", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-15T01:32:30.066 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "inject_early_sigterm": "false", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "jaeger_tracing_enable": "false", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_aio": "true", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_align_min_size": "65536", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_block_align": "true", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_block_size": "4096", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_dio": "true", 2024-09-15T01:32:30.067 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_discard": "false", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_force_aio": "false", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_ignore_corruption": "false", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_max_write_bytes": "10485760", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_max_write_entries": "100", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_replay_from": "0", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_write_header_frequency": "0", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_zero_on_create": "false", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journaler_prefetch_periods": "10", 2024-09-15T01:32:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: "journaler_prezero_periods": "5", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "journaler_write_head_interval": "15", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "keyfile": "", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_backend": "rocksdb", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_default_stripe_size": "65536", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_fsck_on_mount": "false", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_max_bytes": "67108864", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_max_ops": "512", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_nid_prealloc": "1024", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_onode_map_size": "1024", 2024-09-15T01:32:30.069 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_sync_submit_transaction": "false", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_sync_transaction": "false", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_block_size": "65536", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_bloom_size": "0", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_cache_size": "536870912", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_compact_on_mount": "false", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_compression": "false", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_log": "", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_max_open_files": "0", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_paranoid": "false", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-15T01:32:30.070 INFO:teuthology.orchestra.run.smithi042.stdout: "librados_thread_count": "2", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "lockdep": "false", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "lockdep_force_backtrace": "false", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_coarse_timestamps": "true", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_flush_on_exit": "false", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_graylog_host": "127.0.0.1", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_graylog_port": "12201", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_max_new": "1000", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_max_recent": "10000", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_stderr_prefix": "", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_stop_at_utilization": "0.970000", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": "true", 2024-09-15T01:32:30.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_graylog": "false", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_journald": "false", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_stderr": "false", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_syslog": "false", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "max_rotating_auth_attempts": "10", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_action_on_write_error": "1", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_alternate_name_max": "8192", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_asio_thread_count": "2", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_export_pin": "true", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_dirs": "true", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_interval": "5", 2024-09-15T01:32:30.072 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_interval": "10", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_max": "-1", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_max_until": "-1", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_merge_size": "50", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_midchunk": "0.300000", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_min_start": "0.200000", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_minchunk": "0.001000", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_mode": "0", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_need_max": "1.200000", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_need_min": "0.800000", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-15T01:32:30.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_bits": "3", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_size": "10000", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_target_decay": "10.000000", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_beacon_grace": "15.000000", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_beacon_interval": "4.000000", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_mid": "0.700000", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_release_free_interval": "10", 2024-09-15T01:32:30.074 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_reservation": "0.050000", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_trim_interval": "1", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_trim_threshold": "262144", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_client_prealloc_inos": "1000", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_connect_bootstrapping": "false", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_damage_table_max_entries": "10000", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_auth_pins": "false", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_frag": "false", 2024-09-15T01:32:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_scatterstat": "false", 2024-09-15T01:32:30.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_subtrees": "false", 2024-09-15T01:32:30.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_decay_halflife": "5.000000", 2024-09-15T01:32:30.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_default_dir_hash": "2", 2024-09-15T01:32:30.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_defer_session_stale": "true", 2024-09-15T01:32:30.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_deny_all_reconnect": "false", 2024-09-15T01:32:30.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_keys_per_op": "16384", 2024-09-15T01:32:30.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_max_commit_size": "10", 2024-09-15T01:32:30.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_max_entries": "0", 2024-09-15T01:32:30.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-15T01:32:30.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-15T01:32:30.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_on_map": "false", 2024-09-15T01:32:30.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_early_reply": "true", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_enable_op_tracker": "true", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_enforce_unique_name": "true", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_random": "true", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-15T01:32:30.077 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_health_summarize_threshold": "10", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_health_dummy": "false", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_migrator_session_race": "false", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_join_fs": "", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_journal_format": "1", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_create_at": "0", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_export_at": "0", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_import_at": "0", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_journal_at": "0", 2024-09-15T01:32:30.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_journal_expire_at": "0", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_journal_replay_at": "0", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_link_at": "0", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_mdstable_at": "0", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_openc_at": "0", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_rename_at": "0", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_events_per_segment": "1024", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_max_events": "-1", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_max_segments": "128", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_pause": "false", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_segment_size": "0", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_warn_factor": "2.000000", 2024-09-15T01:32:30.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_caps_per_client": "1048576", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_completed_flushes": "100000", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_completed_requests": "100000", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_export_size": "20971520", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_file_recover": "32", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_purge_files": "64", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_purge_ops": "8192", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_snaps_per_dir": "100", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_xattr_pairs_size": "65536", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_metrics_update_interval": "2", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_min_caps_per_client": "100", 2024-09-15T01:32:30.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_min_caps_working_set": "10000", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_numa_node": "-1", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_complaint_time": "30.000000", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_duration": "600", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_size": "20", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_slow_op_size": "20", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_log_threshold": "5", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_ping_grace": "15", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_ping_interval": "5", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-15T01:32:30.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_max_caps": "30000", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_warning_threshold": "262144", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_replay_interval": "1.000000", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_root_ino_gid": "0", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_root_ino_uid": "0", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-15T01:32:30.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_shutdown_check": "0", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_skip_ino": "0", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_snap_max_uid": "4294967294", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_snap_min_uid": "0", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_snap_rstat": "false", 2024-09-15T01:32:30.083 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_standby_replay_damaged": "false", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_symlink_recovery": "true", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_thrash_exports": "0", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_thrash_fragments": "0", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_tick_interval": "5.000000", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_valgrind_exit": "false", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_verify_backtrace": "1", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_verify_scatter": "false", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_wipe_sessions": "false", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool_debug": "false", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-15T01:32:30.084 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_device_bytes": "1073741824", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_page_set": "false", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_page_size": "65536", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_client_bytes": "134217728", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_client_messages": "512", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_disabled_modules": "diskprediction_local", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_max_pg_num_change": "128", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mds_bytes": "134217728", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mds_messages": "128", 2024-09-15T01:32:30.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mon_bytes": "134217728", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mon_messages": "128", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_osd_bytes": "536870912", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_osd_messages": "8192", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_pool": "true", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_standby_modules": "true", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_stats_period": "5", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_stats_threshold": "5", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_tick_period": "2", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-15T01:32:30.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_allow_pool_delete": "true", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_allow_pool_size_one": "true", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_auth_validate_all_caps": "true", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_bytes": "104857600", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_directed_command_retry": "5", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_parallel": "3", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_log_interval": "1.000000", 2024-09-15T01:32:30.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_ping_interval": "10.000000", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_level": "debug", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_file": "true", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_journald": "false", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-15T01:32:30.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_compact_on_bootstrap": "false", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_compact_on_start": "false", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_compact_on_trim": "true", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cpu_threads": "4", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_crush_min_required_version": "hammer", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_daemon_bytes": "419430400", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data_avail_crit": "5", 2024-09-15T01:32:30.089 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data_avail_warn": "5", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data_size_warn": "16106127360", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_dump_json": "false", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_dump_transactions": "false", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_extra_checks": "false", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_require_pacific": "false", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_require_quincy": "false", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-15T01:32:30.090 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_down_mkfs_grace": "120", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_election_default_strategy": "3", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_election_timeout": "5.000000", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_elector_ping_divisor": "2", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_enable_op_tracker": "true", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_fake_pool_delete": "false", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_force_quorum_join": "false", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_globalid_prealloc": "10000", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_detail_to_clog": "false", 2024-09-15T01:32:30.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_log_update_period": "5", 2024-09-15T01:32:30.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_max_detail": "50", 2024-09-15T01:32:30.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_to_clog": "true", 2024-09-15T01:32:30.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_to_clog_interval": "600", 2024-09-15T01:32:30.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-15T01:32:30.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_host": "172.21.15.42,172.21.15.115,172.21.15.152", 2024-09-15T01:32:30.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_host_override": "", 2024-09-15T01:32:30.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_initial_members": "", 2024-09-15T01:32:30.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-15T01:32:30.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-15T01:32:30.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-15T01:32:30.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-15T01:32:30.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_lease": "5.000000", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_log_full_interval": "50", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_log_max": "10000", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_log_max_summary": "50", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_log_epochs": "500", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_osd": "10000", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_pg_per_osd": "10000", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_pool_pg_num": "65536", 2024-09-15T01:32:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mds_force_trim_to": "0", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mds_skip_sanity": "false", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_memory_autotune": "true", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_memory_target": "2147483648", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_beacon_grace": "30", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_digest_period": "5", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_inactive_grace": "60", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_min_osdmap_epochs": "500", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_complaint_time": "30", 2024-09-15T01:32:30.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_duration": "600", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_size": "20", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_slow_op_size": "20", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_log_threshold": "5", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_auto_mark_in": "false", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_cache_size": "500", 2024-09-15T01:32:30.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_down_out_interval": "600", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_force_trim_to": "0", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-15T01:32:30.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_min_down_reporters": "2", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_report_timeout": "900", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-15T01:32:30.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_stuck_threshold": "60", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_probe_timeout": "2.000000", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_max_change": "0.050000", 2024-09-15T01:32:30.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_max_osds": "4", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_interval": "300", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_max_keys": "100", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_timeout": "300", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_session_timeout": "300", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_smart_report_timeout": "5", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stat_smooth_intervals": "6", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_pool_min_size": "2", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_pool_size": "4", 2024-09-15T01:32:30.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_debug": "false", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_provider_kill_at": "0", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_requester_kill_at": "0", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_timeout": "60.000000", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_target_pg_per_osd": "100", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_tick_interval": "5", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_timecheck_interval": "300.000000", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_older_version_delay": "604800", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-15T01:32:30.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_misplaced": "false", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_older_version": "true", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-15T01:32:30.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": "", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_op_threads": "3", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_cm": "false", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_device_name": "", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_dscp": "96", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_local_gid": "", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-15T01:32:30.102 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_port_num": "1", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_sl": "3", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_support_srq": "true", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_type": "ib", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_reap_threshold": "5", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_before_connect": "false", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_ipv4": "true", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_ipv6": "false", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_msgr1": "true", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_msgr2": "true", 2024-09-15T01:32:30.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_port_max": "7568", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_port_min": "6800", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_retry_count": "3", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_retry_delay": "5", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_client": "false", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_mds": "false", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_mgr": "false", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_mon": "false", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_osd": "false", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_client_mode": "crc secure", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_cluster_mode": "crc secure", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_cluster_type": "", 2024-09-15T01:32:30.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_compress_secure": "false", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_connection_idle_timeout": "900", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_connection_ready_timeout": "10", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_crc_data": "true", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_crc_header": "true", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_bad_msg": "false", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_bug": "true", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_old_message": "true", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_skipped_message": "false", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_coremask": "0xF", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-15T01:32:30.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_enable_tso": "true", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_hugepages": "", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_lro": "true", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_memory_channel": "4", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_pmd": "", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_port_id": "0", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-15T01:32:30.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dump_on_send": "false", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_initial_backoff": "0.200000", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_delay_max": "1.000000", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_delay_probability": "0.005000", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_delay_type": "osd", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_internal_delays": "0.002000", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_socket_failures": "2500", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_learn_addr_from_peer": "true", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_max_accept_failures": "4", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_max_backoff": "15.000000", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_mon_client_mode": "secure crc", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-15T01:32:30.107 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_mon_service_mode": "secure crc", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_osd_compress_min_size": "1024", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_osd_compress_mode": "none", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_pq_min_cost": "65536", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_public_type": "", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_service_mode": "crc secure", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_listen_backlog": "512", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_nodelay": "true", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_rcvbuf": "0", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_type": "async+posix", 2024-09-15T01:32:30.108 INFO:teuthology.orchestra.run.smithi042.stdout: "no_config_file": "false", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_completion_locks_per_session": "32", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_inflight_ops": "1024", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_tick_interval": "5.000000", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_timeout": "10.000000", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "objectstore_blackhole": "false", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "openssl_engine_opts": "", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_delay_time": "5.000000", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_hist_halflife": "1000", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_max_low_ops": "2", 2024-09-15T01:32:30.109 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_max_ops": "4", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_slop": "0.020000", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_asio_thread_count": "2", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_async_recovery_min_cost": "1", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backfill_scan_max": "512", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backfill_scan_min": "64", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backoff_on_degraded": "false", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backoff_on_peering": "false", 2024-09-15T01:32:30.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backoff_on_unfound": "true", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_beacon_report_interval": "300", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_duration": "30", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_max_block_size": "67108864", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_blkin_trace_all": "false", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_check_for_log_corruption": "false", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T01:32:30.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_update_on_start": "true", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_message_cap": "256", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_message_size_cap": "524288000", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_op_priority": "63", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_watch_timeout": "30", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_command_max_records": "256", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_compact_on_start": "false", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_update_on_start": "true", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_update_weight_set": "true", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-15T01:32:30.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-15T01:32:30.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-15T01:32:30.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-15T01:32:30.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-15T01:32:30.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_feed_pullee": "-1", 2024-09-15T01:32:30.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-15T01:32:30.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-15T01:32:30.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-15T01:32:30.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_misdirected_ops": "false", 2024-09-15T01:32:30.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_no_acting_change": "false", 2024-09-15T01:32:30.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_no_purge_strays": "false", 2024-09-15T01:32:30.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_op_order": "false", 2024-09-15T01:32:30.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_shutdown": "false", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_snaps": "false", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_keys": "1024", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-15T01:32:30.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_stride": "524288", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_default_notify_timeout": "30", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep": "0.000000", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_discard_disconnected_ops": "true", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_enable_op_tracker": "true", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-15T01:32:30.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_info": "true", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_shutdown": "true", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_function_tracing": "false", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_grace": "20", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_interval": "6", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-15T01:32:30.116 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_min_peers": "10", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_min_size": "2000", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_stale": "600", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_hit_set_max_size": "100000", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_hit_set_min_size": "1000", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_journal_size": "5120", 2024-09-15T01:32:30.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_kill_backfill_at": "0", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_cache_size": "50", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_dedup": "true", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_message_max": "40", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_share_max_epochs": "40", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_attr_name_len": "100", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_attr_size": "0", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_backfills": "1", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_markdown_count": "5", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_markdown_period": "600", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_object_name_len": "2048", 2024-09-15T01:32:30.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_object_namespace_len": "256", 2024-09-15T01:32:30.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_object_size": "134217728", 2024-09-15T01:32:30.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-15T01:32:30.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-15T01:32:30.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pg_blocked_by": "16", 2024-09-15T01:32:30.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pg_log_entries": "10000", 2024-09-15T01:32:30.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-15T01:32:30.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pgls": "1024", 2024-09-15T01:32:30.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_push_cost": "8388608", 2024-09-15T01:32:30.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_push_objects": "10", 2024-09-15T01:32:30.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_scrubs": "3", 2024-09-15T01:32:30.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-15T01:32:30.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_trimming_pgs": "2", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_write_op_reply_len": "64", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_write_size": "90", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_profile": "balanced", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-15T01:32:30.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_base": "805306368", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_cache_min": "134217728", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-15T01:32:30.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_target": "4294967296", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_target_autotune": "false", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_min_pg_log_entries": "250", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_min_recovery_priority": "0", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_report_interval": "5", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_num_cache_shards": "32", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_num_op_tracker_shard": "32", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_numa_auto_affinity": "true", 2024-09-15T01:32:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_numa_node": "-1", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_numa_prefer_iface": "true", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_object_clean_region_max_num_intervals": "1000", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objecter_finishers": "1", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objectstore": "bluestore", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objectstore_fuse": "false", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objectstore_tracing": "false", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_complaint_time": "30.000000", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_duration": "600", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_size": "20", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_slow_op_size": "20", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_log_threshold": "5", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_shards": "0", 2024-09-15T01:32:30.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_shards_hdd": "5", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_shards_ssd": "8", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_pq_min_cost": "65536", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_queue_cut_off": "high", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_thread_timeout": "15", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_open_classes_on_start": "true", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_os_flags": "0", 2024-09-15T01:32:30.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_peering_op_priority": "255", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_delete_cost": "1048576", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_delete_priority": "5", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_log_trim_max": "10000", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_log_trim_min": "100", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-15T01:32:30.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flags": "0", 2024-09-15T01:32:30.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_min_size": "0", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_pg_num": "32", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_pgp_num": "0", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_size": "2", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_type": "replicated", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_push_per_object_cost": "1000", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recover_clone_overlap": "true", 2024-09-15T01:32:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_cost": "20971520", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_active": "0", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_single_start": "1", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_op_priority": "3", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_priority": "5", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep": "0.000000", 2024-09-15T01:32:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_repair_during_recovery": "false", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_requested_scrub_priority": "120", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_auto_repair": "false", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_begin_hour": "0", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_begin_week_day": "0", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_chunk_max": "25", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_chunk_min": "5", 2024-09-15T01:32:30.129 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_cost": "52428800", 2024-09-15T01:32:30.130 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_during_recovery": "false", 2024-09-15T01:32:30.130 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_end_hour": "0", 2024-09-15T01:32:30.130 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_end_week_day": "0", 2024-09-15T01:32:30.130 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_event_cost": "4096", 2024-09-15T01:32:30.130 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-15T01:32:30.130 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-15T01:32:30.130 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_invalid_stats": "true", 2024-09-15T01:32:30.130 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-15T01:32:30.130 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-15T01:32:30.130 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_max_preemptions": "5", 2024-09-15T01:32:30.130 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-15T01:32:30.130 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_priority": "5", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_sleep": "0.000000", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_skip_data_digest": "false", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_smart_report_timeout": "5", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_cost": "1048576", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_priority": "5", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_target_transaction_size": "30", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-15T01:32:30.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tracing": "false", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_use_stale_snap": "false", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osdc_blkin_trace_all": "false", 2024-09-15T01:32:30.132 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_kill_at": "0", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_max_join_drift": "10", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_min": "500", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_min_wait": "0.050000", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_propose_interval": "1.000000", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_service_trim_max": "500", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_service_trim_min": "250", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_stash_full_interval": "25", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_trim_max": "500", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_trim_min": "250", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "perf": "true", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-15T01:32:30.133 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "-", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrv": "", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "public_bind_addr": "-", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "public_network": "", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "public_network_interface": "", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "qat_compressor_enabled": "false", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_mon_op_timeout": "0", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_osd_op_timeout": "0", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_tracing": "false", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_atime_update_interval": "60", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_balance_parent_reads": "false", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_balance_snap_reads": "false", 2024-09-15T01:32:30.134 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_blkin_trace_all": "false", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache": "true", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_policy": "writearound", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_size": "33554432", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_clone_copy_on_read": "false", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_compression_hint": "none", 2024-09-15T01:32:30.135 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_concurrent_management_ops": "10", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_clone_format": "auto", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_data_pool": "", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_features": "61", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_format": "2", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_map_options": "", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_order": "22", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_pool": "rbd", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_stripe_count": "0", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_stripe_unit": "0", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-15T01:32:30.136 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_enable_alloc_hint": "true", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_io_scheduler": "simple", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_order": "24", 2024-09-15T01:32:30.137 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_pool": "", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_splay_width": "4", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_localize_parent_reads": "false", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_localize_snap_reads": "false", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-15T01:32:30.138 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-15T01:32:30.139 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mtime_update_interval": "60", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_non_blocking_aio": "true", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_op_thread_timeout": "60", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_op_threads": "1", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_parent_cache_enabled": "false", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-15T01:32:30.140 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_plugins": "", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_bps_burst": "0", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_bps_limit": "0", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_exclude_ops": "0", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_iops_burst": "0", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_iops_limit": "0", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-15T01:32:30.141 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_read_from_replica_policy": "default", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-15T01:32:30.142 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_skip_partial_discard": "true", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_tracing": "false", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_validate_names": "true", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_validate_pool": "true", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "restapi_base_url": "", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "restapi_log_level": "", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_acl_grants_max_num": "100", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_admin_entry": "admin", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_backend_store": "rados", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_barbican_url": "", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_beast_enable_async": "true", 2024-09-15T01:32:30.143 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cache_enabled": "true", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cache_expiry_interval": "900", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cache_lru_size": "10000", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_content_length_compat": "false", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_copy_obj_progress": "true", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cors_rules_max_num": "100", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cross_domain_policy": "", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-15T01:32:30.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_addr": "", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_password": "", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_username": "", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_require_ssl": "true", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-15T01:32:30.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_addr": "", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_auth": "token", 2024-09-15T01:32:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_namespace": "", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_prefix": "", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_token_file": "", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_buffersize": "524288", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_low_speed_time": "300", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-15T01:32:30.147 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_changes_size": "1000", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_num_shards": "128", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_window": "30", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_notify_interval_msec": "200", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-15T01:32:30.148 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-15T01:32:30.149 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dns_name": "", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dns_s3website_name": "", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dynamic_resharding": "true", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_gc_threads": "true", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_lc_threads": "true", 2024-09-15T01:32:30.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_ops_log": "false", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_quota_threads": "true", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_static_website": "false", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_usage_log": "false", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enforce_swift_acls": "true", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_exit_timeout_secs": "120", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_expose_bucket": "false", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_extended_http_attrs": "", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_frontends": "beast port=7480", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_deferred": "50", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_objs": "32", 2024-09-15T01:32:30.151 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-15T01:32:30.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-15T01:32:30.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-15T01:32:30.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-15T01:32:30.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_processor_period": "3600", 2024-09-15T01:32:30.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-15T01:32:30.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-15T01:32:30.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-15T01:32:30.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-15T01:32:30.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_init_timeout": "300", 2024-09-15T01:32:30.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-15T01:32:30.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_kafka_connection_idle": "300", 2024-09-15T01:32:30.152 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_domain": "", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_password": "", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_password_path": "", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_project": "", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_tenant": "", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_token": "", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_token_path": "", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_user": "", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_api_version": "2", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_domain": "", 2024-09-15T01:32:30.153 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_password": "", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_project": "", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_user": "", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_url": "", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_debug_interval": "-1", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_lock_max_time": "90", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_objs": "32", 2024-09-15T01:32:30.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_rules": "1000", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_worker": "3", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_thread_delay": "0", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_dnattr": "uid", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_searchfilter": "", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_uri": "", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_http_headers": "", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-15T01:32:30.155 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_object_name_utc": "false", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_attr_name_len": "0", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_attr_size": "0", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_chunk_size": "4194304", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_listing_results": "1000", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_notify_retries": "10", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_put_param_size": "1048576", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_put_size": "5368709120", 2024-09-15T01:32:30.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_slo_entries": "1000", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_md_log_max_shards": "64", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_mp_lock_max_time": "600", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_max_gc": "300", 2024-09-15T01:32:30.157 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_num_async_rados_threads": "32", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_num_control_oids": "8", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_numa_node": "-1", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_objexp_chunk_size": "100", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_objexp_gc_interval": "600", 2024-09-15T01:32:30.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_op_thread_timeout": "600", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_op_tracing": "false", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_opa_token": "", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_opa_url": "", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_opa_verify_ssl": "true", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_file_path": "", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_rados": "true", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_socket_path": "", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-15T01:32:30.159 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_push_interval": "2.000000", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_print_continue": "true", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rados_tracing": "false", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_realm": "", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_realm_id": "", 2024-09-15T01:32:30.160 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_region": "", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_request_uri": "", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_batch_size": "64", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_max_aio": "128", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_num_logs": "16", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_thread_interval": "600", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_resolve_cname": "false", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-15T01:32:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_run_sync_thread": "true", 2024-09-15T01:32:30.162 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-15T01:32:30.162 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-15T01:32:30.162 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-15T01:32:30.162 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-15T01:32:30.162 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-15T01:32:30.162 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-15T01:32:30.162 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-15T01:32:30.162 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-15T01:32:30.162 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-15T01:32:30.162 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_scheduler_type": "throttler", 2024-09-15T01:32:30.162 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_script_uri": "", 2024-09-15T01:32:30.162 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_service_provider_name": "", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_client_id": "", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_client_secret": "", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_entry": "sts", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_key": "sts", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_min_session_duration": "900", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_token_introspection_url": "", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_account_in_url": "false", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_auth_entry": "auth", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_auth_url": "", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_custom_header": "", 2024-09-15T01:32:30.163 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_need_stats": "true", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_tenant_name": "", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_token_expiration": "86400", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_url": "", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_url_prefix": "swift", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_lease_period": "120", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-15T01:32:30.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_thread_pool_size": "512", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_comment": "", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_createby": "", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_encoding": "", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_flag": "false", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_origin": "", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_tracker": "", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_trust_forwarded_https": "false", 2024-09-15T01:32:30.165 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_max_shards": "32", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_max_user_shards": "1", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_use_opa_authz": "false", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_max_buckets": "1000", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_unique_email": "true", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_verify_ssl": "true", 2024-09-15T01:32:30.166 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zone": "", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zone_id": "", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zonegroup": "", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zonegroup_id": "", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_block_size": "4096", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-15T01:32:30.167 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_size": "536870912", 2024-09-15T01:32:30.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-15T01:32:30.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-15T01:32:30.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-15T01:32:30.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-15T01:32:30.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-15T01:32:30.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-15T01:32:30.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-15T01:32:30.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-15T01:32:30.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_index_type": "binary_search", 2024-09-15T01:32:30.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-15T01:32:30.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-15T01:32:30.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_partition_filters": "false", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_perf": "false", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "run_dir": "/var/run/ceph", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_block_create": "true", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_cache_lru_size": "67108864", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_default_max_object_size": "16777216", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_device_size": "107374182400", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_batch_capacity": "16", 2024-09-15T01:32:30.169 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-15T01:32:30.170 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-15T01:32:30.170 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-15T01:32:30.170 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_segment_size": "67108864", 2024-09-15T01:32:30.170 INFO:teuthology.orchestra.run.smithi042.stdout: "setgroup": "", 2024-09-15T01:32:30.170 INFO:teuthology.orchestra.run.smithi042.stdout: "setuser": "", 2024-09-15T01:32:30.170 INFO:teuthology.orchestra.run.smithi042.stdout: "setuser_match_path": "", 2024-09-15T01:32:30.170 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-15T01:32:30.170 INFO:teuthology.orchestra.run.smithi042.stdout: "thp": "false", 2024-09-15T01:32:30.170 INFO:teuthology.orchestra.run.smithi042.stdout: "threadpool_default_timeout": "60", 2024-09-15T01:32:30.170 INFO:teuthology.orchestra.run.smithi042.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-15T01:32:30.170 INFO:teuthology.orchestra.run.smithi042.stdout: "throttler_perf_counter": "true" 2024-09-15T01:32:30.170 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:30.173 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-15T01:32:30.295 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a: {} 2024-09-15T01:32:30.296 INFO:teuthology.orchestra.run.smithi042.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T01:32:30.309 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b: {} 2024-09-15T01:32:30.310 INFO:teuthology.orchestra.run.smithi042.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T01:32:30.324 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c: {} 2024-09-15T01:32:30.324 INFO:teuthology.orchestra.run.smithi042.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T01:32:30.335 DEBUG:teuthology.orchestra.run.smithi042:> type ceph-objectstore-tool 2024-09-15T01:32:30.350 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T01:32:30.350 DEBUG:teuthology.orchestra.run.smithi115:> type ceph-objectstore-tool 2024-09-15T01:32:30.370 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T01:32:30.370 DEBUG:teuthology.orchestra.run.smithi175:> type ceph-objectstore-tool 2024-09-15T01:32:30.388 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T01:32:30.389 DEBUG:teuthology.orchestra.run.smithi152:> type ceph-objectstore-tool 2024-09-15T01:32:30.407 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T01:32:30.407 INFO:teuthology.run_tasks:Running task rados... 2024-09-15T01:32:30.418 INFO:tasks.rados:Beginning rados... 2024-09-15T01:32:30.418 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-15T01:32:30.419 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd sloppy crc': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'osd objectstore': 'filestore', 'osd sloppy crc': True}}, 'filestore': True, 'fs': 'xfs'}, 'install': {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-15T01:32:30.419 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-15T01:32:30.419 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-09-15T01:32:30.426 INFO:tasks.rados:joining rados 2024-09-15T01:32:30.426 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-15T01:32:30.427 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-15T01:32:30.427 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:32:30.427 INFO:tasks.thrashosds.thrasher:Reweighting osd 11 to 0.3046597180048667 2024-09-15T01:32:30.427 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 0.3046597180048667 2024-09-15T01:32:30.427 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-15T01:32:30.428 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-15T01:32:30.428 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:32:30.428 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-15T01:32:30.428 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:32:30.429 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-15T01:32:30.429 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:32:30.429 INFO:tasks.rados:clients are ['client.0'] 2024-09-15T01:32:30.429 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set shecprofile c=2 crush-failure-domain=osd k=4 m=3 name=shecprofile plugin=shec 2024-09-15T01:32:30.431 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:30.426+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:30.530 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:30.528+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:30.541 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:32:30.541 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:30.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:32:30.573 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:32:30.573 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:32:30.597 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:32:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:32:30.618 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:32:30.618 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:32:30.632 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:30.630+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:30.637 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:32:30.637 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:32:30.658 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:32:30.658 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:32:30.678 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:32:30.678 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:32:30.700 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:30.701 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:30.712 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:32:30.712 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:32:30.713 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:32:30.733 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:30.731+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:30.747 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:32:30.747 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:32:30.769 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:32:30.770 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:32:30.792 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:32:30.792 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:32:30.812 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:32:30.812 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:32:30.831 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:32:30.832 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:32:30.833 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:30.830+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:30.851 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:32:30.852 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:32:30.871 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:32:30.872 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:32:30.877 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:30.877 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:30.886 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:32:30.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:32:30.889+0000 7f0c4b7fe640 0 -- 172.21.15.42:0/74150323 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7f0c4c192510 msgr2=0x7f0c4c194910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:32:30.897 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:32:30.917 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:32:30.917 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:32:30.934 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:30.932+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:30.992 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:30.992 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:31.000 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:32:31.035 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:31.033+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:31.064 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.11 to 0.30466 (4dfe) 2024-09-15T01:32:31.064 INFO:teuthology.orchestra.run.smithi042.stderr:load: shec 2024-09-15T01:32:31.064 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-15T01:32:31.080 INFO:tasks.rados:starting run 0 out of 1 2024-09-15T01:32:31.080 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-15T01:32:31.080 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure shecprofile 2024-09-15T01:32:31.110 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:31.110 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:31.119 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:32:31.135 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:31.134+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:31.236 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:31.235+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:31.304 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:31.304 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:31.317 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:32:31.337 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:31.335+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:31.419 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:31.419 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:31.428 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:32:31.438 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:31.437+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:31.528 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:31.528 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:31.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:32:31.538 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:31.537+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:31.639 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:31.639 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:31.640 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:31.638+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:31.647 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-15T01:32:31.740 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:31.739+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:31.748 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:31.748 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:31.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-15T01:32:31.841 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:31.840+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:31.857 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:31.858 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:31.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-15T01:32:31.942 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:31.939+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:31.967 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:31.967 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:31.976 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:32:32.042 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:32.041+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:32.084 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:32.084 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:32.093 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:32:32.144 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:32.141+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:32.191 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:32.191 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:32.199 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:32:32.245 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:32.244+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:32.335 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:32.335 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:32.344 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:32:32.345 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:32.343+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:32.440 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:32.440 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:32.446 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:32.445+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:32.448 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:32:32.544 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:32.544 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:32.546 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:32.544+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:32.552 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:32:32.648 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:32.646+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:32.687 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:32.688 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:32.696 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:32:32.749 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:32.747+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:32.793 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:32.793 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:32.801 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:32:32.850 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:32.848+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:32.929 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:32:32.937 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:32.937 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:32.946 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:32:32.989 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:32.987+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:33.042 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:33.042 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:33.051 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-15T01:32:33.051 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:33.048+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:33.069 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:32:33.069 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:32:33.086 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T01:32:33.089 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:32:33.089 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:32:33.109 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:32:33.109 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:32:33.129 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:32:33.129 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:32:33.149 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:32:33.149 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:32:33.151 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:33.149+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'unique_pool_0' created 2024-09-15T01:32:33.175 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:32:33.176 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:32:33.179 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-15T01:32:33.193 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:33.193 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:33.196 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:32:33.197 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:32:33.202 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-15T01:32:33.218 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:32:33.218 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:32:33.237 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:32:33.237 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:32:33.256 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:32:33.256 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:32:33.276 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:32:33.276 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:32:33.286 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:33.284+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:33.295 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:32:33.295 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:32:33.315 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:32:33.315 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:32:33.335 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:32:33.335 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:32:33.340 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:33.341 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:33.341 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:32:33.341 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:32:33.341 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:33.349 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-15T01:32:33.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:32:33.351+0000 7facc6ffd640 0 -- 172.21.15.42:0/3787082893 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7facc8194b70 msgr2=0x7facc8196f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:32:33.390 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:33.388+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:33.453 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:33.452+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:33.489 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:33.489 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:32:33.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:32:33.489 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:32:33.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.489 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.489 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:32:33.489 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.005953158000001, 2024-09-15T01:32:33.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27210968699999999, 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:32:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479166+0000", 2024-09-15T01:32:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4729999999999999e-06 2024-09-15T01:32:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479172+0000", 2024-09-15T01:32:33.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7460000000000003e-06 2024-09-15T01:32:33.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479186+0000", 2024-09-15T01:32:33.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4011e-05 2024-09-15T01:32:33.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479211+0000", 2024-09-15T01:32:33.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4658e-05 2024-09-15T01:32:33.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479249+0000", 2024-09-15T01:32:33.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7706000000000001e-05 2024-09-15T01:32:33.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-15T01:32:33.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479255+0000", 2024-09-15T01:32:33.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9669999999999998e-06 2024-09-15T01:32:33.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750042+0000", 2024-09-15T01:32:33.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27078760899999998 2024-09-15T01:32:33.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750204+0000", 2024-09-15T01:32:33.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016171800000000001 2024-09-15T01:32:33.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750337+0000", 2024-09-15T01:32:33.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000132674 2024-09-15T01:32:33.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750470+0000", 2024-09-15T01:32:33.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013314699999999999 2024-09-15T01:32:33.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750485+0000", 2024-09-15T01:32:33.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5288e-05 2024-09-15T01:32:33.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750515+0000", 2024-09-15T01:32:33.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0179e-05 2024-09-15T01:32:33.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750582+0000", 2024-09-15T01:32:33.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6890999999999994e-05 2024-09-15T01:32:33.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751212+0000", 2024-09-15T01:32:33.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062971900000000005 2024-09-15T01:32:33.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751249+0000", 2024-09-15T01:32:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7072e-05 2024-09-15T01:32:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751274+0000", 2024-09-15T01:32:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4828999999999999e-05 2024-09-15T01:32:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.499 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.145983+0000", 2024-09-15T01:32:33.499 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.339134439999999, 2024-09-15T01:32:33.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043641740000000002, 2024-09-15T01:32:33.499 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.499 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.499 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49 2024-09-15T01:32:33.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.499 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.145983+0000", 2024-09-15T01:32:33.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.145983+0000", 2024-09-15T01:32:33.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.145983+0000", 2024-09-15T01:32:33.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-15T01:32:33.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.145986+0000", 2024-09-15T01:32:33.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6359999999999998e-06 2024-09-15T01:32:33.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.145991+0000", 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1170000000000003e-06 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.145994+0000", 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149173+0000", 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031784869999999998 2024-09-15T01:32:33.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149226+0000", 2024-09-15T01:32:33.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3553e-05 2024-09-15T01:32:33.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149300+0000", 2024-09-15T01:32:33.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3195999999999995e-05 2024-09-15T01:32:33.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149417+0000", 2024-09-15T01:32:33.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011720199999999999 2024-09-15T01:32:33.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149425+0000", 2024-09-15T01:32:33.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2020000000000004e-06 2024-09-15T01:32:33.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149458+0000", 2024-09-15T01:32:33.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3438000000000001e-05 2024-09-15T01:32:33.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149587+0000", 2024-09-15T01:32:33.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012824200000000001 2024-09-15T01:32:33.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150289+0000", 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070282600000000004 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150322+0000", 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3009999999999997e-05 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150347+0000", 2024-09-15T01:32:33.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4501000000000001e-05 2024-09-15T01:32:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.146017+0000", 2024-09-15T01:32:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.339100753, 2024-09-15T01:32:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048023450000000004, 2024-09-15T01:32:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.508 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 50 2024-09-15T01:32:33.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.508 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146017+0000", 2024-09-15T01:32:33.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146017+0000", 2024-09-15T01:32:33.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146017+0000", 2024-09-15T01:32:33.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7899999999999998e-07 2024-09-15T01:32:33.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146074+0000", 2024-09-15T01:32:33.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6870000000000003e-05 2024-09-15T01:32:33.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146082+0000", 2024-09-15T01:32:33.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.011e-06 2024-09-15T01:32:33.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146087+0000", 2024-09-15T01:32:33.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1080000000000001e-06 2024-09-15T01:32:33.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149687+0000", 2024-09-15T01:32:33.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003599667 2024-09-15T01:32:33.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149763+0000", 2024-09-15T01:32:33.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6506000000000002e-05 2024-09-15T01:32:33.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149846+0000", 2024-09-15T01:32:33.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2260000000000002e-05 2024-09-15T01:32:33.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149962+0000", 2024-09-15T01:32:33.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116898 2024-09-15T01:32:33.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149973+0000", 2024-09-15T01:32:33.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0200000000000001e-05 2024-09-15T01:32:33.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150038+0000", 2024-09-15T01:32:33.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5068999999999997e-05 2024-09-15T01:32:33.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150377+0000", 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033882399999999999 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150774+0000", 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039736200000000001 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150799+0000", 2024-09-15T01:32:33.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4752999999999998e-05 2024-09-15T01:32:33.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150819+0000", 2024-09-15T01:32:33.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0237999999999998e-05 2024-09-15T01:32:33.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.515 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.515 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.146094+0000", 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.339023036, 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048128390000000002, 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146094+0000", 2024-09-15T01:32:33.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146094+0000", 2024-09-15T01:32:33.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146095+0000", 2024-09-15T01:32:33.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7100000000000002e-07 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146099+0000", 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1030000000000004e-06 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146105+0000", 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9259999999999997e-06 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146111+0000", 2024-09-15T01:32:33.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.998e-06 2024-09-15T01:32:33.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.149998+0000", 2024-09-15T01:32:33.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038868819999999999 2024-09-15T01:32:33.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150057+0000", 2024-09-15T01:32:33.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8981999999999999e-05 2024-09-15T01:32:33.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150133+0000", 2024-09-15T01:32:33.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5978999999999999e-05 2024-09-15T01:32:33.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150229+0000", 2024-09-15T01:32:33.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6088000000000006e-05 2024-09-15T01:32:33.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150235+0000", 2024-09-15T01:32:33.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4919999999999998e-06 2024-09-15T01:32:33.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150258+0000", 2024-09-15T01:32:33.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2365e-05 2024-09-15T01:32:33.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150384+0000", 2024-09-15T01:32:33.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000126367 2024-09-15T01:32:33.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150847+0000", 2024-09-15T01:32:33.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046301399999999998 2024-09-15T01:32:33.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150874+0000", 2024-09-15T01:32:33.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7180000000000001e-05 2024-09-15T01:32:33.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150907+0000", 2024-09-15T01:32:33.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2892000000000003e-05 2024-09-15T01:32:33.524 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.524 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.524 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.524 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.524 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.146116+0000", 2024-09-15T01:32:33.525 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.339001299, 2024-09-15T01:32:33.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049771340000000002, 2024-09-15T01:32:33.525 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.525 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.525 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.525 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.525 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.525 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52 2024-09-15T01:32:33.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.525 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146116+0000", 2024-09-15T01:32:33.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146116+0000", 2024-09-15T01:32:33.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146116+0000", 2024-09-15T01:32:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5199999999999998e-07 2024-09-15T01:32:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146120+0000", 2024-09-15T01:32:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4350000000000001e-06 2024-09-15T01:32:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146126+0000", 2024-09-15T01:32:33.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6579999999999996e-06 2024-09-15T01:32:33.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146130+0000", 2024-09-15T01:32:33.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9709999999999998e-06 2024-09-15T01:32:33.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150406+0000", 2024-09-15T01:32:33.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042754960000000002 2024-09-15T01:32:33.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150503+0000", 2024-09-15T01:32:33.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7405999999999995e-05 2024-09-15T01:32:33.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150614+0000", 2024-09-15T01:32:33.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110476 2024-09-15T01:32:33.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150714+0000", 2024-09-15T01:32:33.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010037799999999999 2024-09-15T01:32:33.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150724+0000", 2024-09-15T01:32:33.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3810000000000005e-06 2024-09-15T01:32:33.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150756+0000", 2024-09-15T01:32:33.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2889999999999999e-05 2024-09-15T01:32:33.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150931+0000", 2024-09-15T01:32:33.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017436700000000001 2024-09-15T01:32:33.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.151051+0000", 2024-09-15T01:32:33.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012015600000000001 2024-09-15T01:32:33.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.151074+0000", 2024-09-15T01:32:33.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3402999999999999e-05 2024-09-15T01:32:33.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.151093+0000", 2024-09-15T01:32:33.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8765000000000001e-05 2024-09-15T01:32:33.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.533 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.533 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.533 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269156+0000", 2024-09-15T01:32:33.533 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.215961619000002, 2024-09-15T01:32:33.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010045861, 2024-09-15T01:32:33.533 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.533 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.534 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.534 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.534 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 72 2024-09-15T01:32:33.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.534 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269156+0000", 2024-09-15T01:32:33.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269156+0000", 2024-09-15T01:32:33.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269156+0000", 2024-09-15T01:32:33.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1199999999999999e-07 2024-09-15T01:32:33.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269218+0000", 2024-09-15T01:32:33.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.19e-05 2024-09-15T01:32:33.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269222+0000", 2024-09-15T01:32:33.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.673e-06 2024-09-15T01:32:33.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269225+0000", 2024-09-15T01:32:33.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.441e-06 2024-09-15T01:32:33.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271274+0000", 2024-09-15T01:32:33.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020488540000000001 2024-09-15T01:32:33.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271323+0000", 2024-09-15T01:32:33.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9258999999999997e-05 2024-09-15T01:32:33.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271394+0000", 2024-09-15T01:32:33.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0745000000000001e-05 2024-09-15T01:32:33.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271483+0000", 2024-09-15T01:32:33.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9605999999999996e-05 2024-09-15T01:32:33.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271491+0000", 2024-09-15T01:32:33.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5800000000000003e-06 2024-09-15T01:32:33.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271554+0000", 2024-09-15T01:32:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3157000000000005e-05 2024-09-15T01:32:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271765+0000", 2024-09-15T01:32:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021087699999999999 2024-09-15T01:32:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279154+0000", 2024-09-15T01:32:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073888729999999998 2024-09-15T01:32:33.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279181+0000", 2024-09-15T01:32:33.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7370000000000001e-05 2024-09-15T01:32:33.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279202+0000", 2024-09-15T01:32:33.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0313999999999999e-05 2024-09-15T01:32:33.541 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.541 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.542 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.542 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269228+0000", 2024-09-15T01:32:33.542 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.215889271999998, 2024-09-15T01:32:33.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010072035, 2024-09-15T01:32:33.542 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.542 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.542 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.542 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.543 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 73 2024-09-15T01:32:33.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.543 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269228+0000", 2024-09-15T01:32:33.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269228+0000", 2024-09-15T01:32:33.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269228+0000", 2024-09-15T01:32:33.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3099999999999999e-07 2024-09-15T01:32:33.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269230+0000", 2024-09-15T01:32:33.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.137e-06 2024-09-15T01:32:33.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269234+0000", 2024-09-15T01:32:33.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0599999999999999e-06 2024-09-15T01:32:33.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269235+0000", 2024-09-15T01:32:33.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.922e-06 2024-09-15T01:32:33.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271530+0000", 2024-09-15T01:32:33.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0022941419999999999 2024-09-15T01:32:33.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271573+0000", 2024-09-15T01:32:33.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3789999999999999e-05 2024-09-15T01:32:33.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271653+0000", 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9827000000000007e-05 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271737+0000", 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3950999999999996e-05 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271742+0000", 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2449999999999998e-06 2024-09-15T01:32:33.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271768+0000", 2024-09-15T01:32:33.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5581000000000001e-05 2024-09-15T01:32:33.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271922+0000", 2024-09-15T01:32:33.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153871 2024-09-15T01:32:33.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279240+0000", 2024-09-15T01:32:33.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073177279999999999 2024-09-15T01:32:33.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279275+0000", 2024-09-15T01:32:33.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5179999999999999e-05 2024-09-15T01:32:33.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279300+0000", 2024-09-15T01:32:33.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.527e-05 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269238+0000", 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.215879440999998, 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010290084999999999, 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.551 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 74 2024-09-15T01:32:33.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.551 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269238+0000", 2024-09-15T01:32:33.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269238+0000", 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269238+0000", 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269240+0000", 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4310000000000001e-06 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269243+0000", 2024-09-15T01:32:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-15T01:32:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269245+0000", 2024-09-15T01:32:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8679999999999999e-06 2024-09-15T01:32:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272014+0000", 2024-09-15T01:32:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027686450000000001 2024-09-15T01:32:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272069+0000", 2024-09-15T01:32:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5031000000000002e-05 2024-09-15T01:32:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272175+0000", 2024-09-15T01:32:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106455 2024-09-15T01:32:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272298+0000", 2024-09-15T01:32:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122891 2024-09-15T01:32:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272304+0000", 2024-09-15T01:32:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3510000000000001e-06 2024-09-15T01:32:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272329+0000", 2024-09-15T01:32:33.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4905999999999998e-05 2024-09-15T01:32:33.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272892+0000", 2024-09-15T01:32:33.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00056393100000000005 2024-09-15T01:32:33.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279488+0000", 2024-09-15T01:32:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065956269999999997 2024-09-15T01:32:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279511+0000", 2024-09-15T01:32:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2527e-05 2024-09-15T01:32:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279528+0000", 2024-09-15T01:32:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7294000000000001e-05 2024-09-15T01:32:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269248+0000", 2024-09-15T01:32:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.215869651999999, 2024-09-15T01:32:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010540288, 2024-09-15T01:32:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.559 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 75 2024-09-15T01:32:33.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.559 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269248+0000", 2024-09-15T01:32:33.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269248+0000", 2024-09-15T01:32:33.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269248+0000", 2024-09-15T01:32:33.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.04e-07 2024-09-15T01:32:33.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269301+0000", 2024-09-15T01:32:33.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2757000000000003e-05 2024-09-15T01:32:33.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269305+0000", 2024-09-15T01:32:33.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3780000000000003e-06 2024-09-15T01:32:33.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269309+0000", 2024-09-15T01:32:33.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.738e-06 2024-09-15T01:32:33.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272346+0000", 2024-09-15T01:32:33.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030372089999999999 2024-09-15T01:32:33.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272414+0000", 2024-09-15T01:32:33.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8232000000000007e-05 2024-09-15T01:32:33.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272543+0000", 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012883799999999999 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272646+0000", 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102948 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272651+0000", 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6879999999999998e-06 2024-09-15T01:32:33.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272703+0000", 2024-09-15T01:32:33.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1901000000000003e-05 2024-09-15T01:32:33.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272897+0000", 2024-09-15T01:32:33.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000194472 2024-09-15T01:32:33.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279745+0000", 2024-09-15T01:32:33.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068477929999999996 2024-09-15T01:32:33.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279770+0000", 2024-09-15T01:32:33.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4820999999999998e-05 2024-09-15T01:32:33.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279788+0000", 2024-09-15T01:32:33.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8309e-05 2024-09-15T01:32:33.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.566 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.566 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.566 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269311+0000", 2024-09-15T01:32:33.566 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.215805819, 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010539287, 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 76 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269311+0000", 2024-09-15T01:32:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269311+0000", 2024-09-15T01:32:33.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269312+0000", 2024-09-15T01:32:33.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9799999999999999e-07 2024-09-15T01:32:33.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269315+0000", 2024-09-15T01:32:33.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7889999999999999e-06 2024-09-15T01:32:33.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269318+0000", 2024-09-15T01:32:33.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0070000000000002e-06 2024-09-15T01:32:33.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269320+0000", 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0540000000000002e-06 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272676+0000", 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033561340000000002 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272730+0000", 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3983999999999997e-05 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272797+0000", 2024-09-15T01:32:33.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7156999999999994e-05 2024-09-15T01:32:33.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272866+0000", 2024-09-15T01:32:33.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9453000000000001e-05 2024-09-15T01:32:33.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272872+0000", 2024-09-15T01:32:33.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9660000000000001e-06 2024-09-15T01:32:33.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272895+0000", 2024-09-15T01:32:33.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2764000000000001e-05 2024-09-15T01:32:33.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272915+0000", 2024-09-15T01:32:33.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0361e-05 2024-09-15T01:32:33.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279812+0000", 2024-09-15T01:32:33.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068969000000000001 2024-09-15T01:32:33.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279833+0000", 2024-09-15T01:32:33.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1055000000000001e-05 2024-09-15T01:32:33.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279851+0000", 2024-09-15T01:32:33.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7365000000000001e-05 2024-09-15T01:32:33.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.574 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.574 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:78 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.574 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.280120+0000", 2024-09-15T01:32:33.575 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.204997627000001, 2024-09-15T01:32:33.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082697889999999996, 2024-09-15T01:32:33.575 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.575 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.575 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78 2024-09-15T01:32:33.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.575 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280120+0000", 2024-09-15T01:32:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280120+0000", 2024-09-15T01:32:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280120+0000", 2024-09-15T01:32:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-09-15T01:32:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280162+0000", 2024-09-15T01:32:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1817999999999998e-05 2024-09-15T01:32:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280167+0000", 2024-09-15T01:32:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-15T01:32:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280170+0000", 2024-09-15T01:32:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5310000000000002e-06 2024-09-15T01:32:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280390+0000", 2024-09-15T01:32:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021957699999999999 2024-09-15T01:32:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280462+0000", 2024-09-15T01:32:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2725000000000006e-05 2024-09-15T01:32:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280551+0000", 2024-09-15T01:32:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8246999999999997e-05 2024-09-15T01:32:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280646+0000", 2024-09-15T01:32:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5063999999999996e-05 2024-09-15T01:32:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280651+0000", 2024-09-15T01:32:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-09-15T01:32:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280706+0000", 2024-09-15T01:32:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4815000000000003e-05 2024-09-15T01:32:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281099+0000", 2024-09-15T01:32:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000392982 2024-09-15T01:32:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288289+0000", 2024-09-15T01:32:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071901609999999996 2024-09-15T01:32:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288336+0000", 2024-09-15T01:32:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7593999999999999e-05 2024-09-15T01:32:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288389+0000", 2024-09-15T01:32:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3022999999999997e-05 2024-09-15T01:32:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.280175+0000", 2024-09-15T01:32:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.204942217999999, 2024-09-15T01:32:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082971350000000006, 2024-09-15T01:32:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 79 2024-09-15T01:32:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280175+0000", 2024-09-15T01:32:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280175+0000", 2024-09-15T01:32:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280175+0000", 2024-09-15T01:32:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-15T01:32:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280217+0000", 2024-09-15T01:32:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1483000000000002e-05 2024-09-15T01:32:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280222+0000", 2024-09-15T01:32:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6260000000000003e-06 2024-09-15T01:32:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280224+0000", 2024-09-15T01:32:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5950000000000001e-06 2024-09-15T01:32:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280700+0000", 2024-09-15T01:32:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047627200000000002 2024-09-15T01:32:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280804+0000", 2024-09-15T01:32:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103742 2024-09-15T01:32:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280917+0000", 2024-09-15T01:32:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112908 2024-09-15T01:32:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281046+0000", 2024-09-15T01:32:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000128555 2024-09-15T01:32:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281059+0000", 2024-09-15T01:32:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.362e-05 2024-09-15T01:32:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281110+0000", 2024-09-15T01:32:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0457000000000002e-05 2024-09-15T01:32:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281671+0000", 2024-09-15T01:32:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00056108199999999997 2024-09-15T01:32:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288422+0000", 2024-09-15T01:32:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00675145 2024-09-15T01:32:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288451+0000", 2024-09-15T01:32:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9133e-05 2024-09-15T01:32:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288472+0000", 2024-09-15T01:32:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0761999999999999e-05 2024-09-15T01:32:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.280228+0000", 2024-09-15T01:32:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.204889681000001, 2024-09-15T01:32:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085368890000000006, 2024-09-15T01:32:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 80 2024-09-15T01:32:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280228+0000", 2024-09-15T01:32:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280228+0000", 2024-09-15T01:32:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280228+0000", 2024-09-15T01:32:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8800000000000003e-07 2024-09-15T01:32:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280281+0000", 2024-09-15T01:32:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2437e-05 2024-09-15T01:32:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280286+0000", 2024-09-15T01:32:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.835e-06 2024-09-15T01:32:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280289+0000", 2024-09-15T01:32:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0790000000000001e-06 2024-09-15T01:32:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281125+0000", 2024-09-15T01:32:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00083510599999999996 2024-09-15T01:32:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281204+0000", 2024-09-15T01:32:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9025999999999999e-05 2024-09-15T01:32:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281350+0000", 2024-09-15T01:32:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000146866 2024-09-15T01:32:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281512+0000", 2024-09-15T01:32:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000161359 2024-09-15T01:32:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281518+0000", 2024-09-15T01:32:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1229999999999997e-06 2024-09-15T01:32:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281573+0000", 2024-09-15T01:32:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.541e-05 2024-09-15T01:32:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281675+0000", 2024-09-15T01:32:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101724 2024-09-15T01:32:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288695+0000", 2024-09-15T01:32:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070196870000000001 2024-09-15T01:32:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288734+0000", 2024-09-15T01:32:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9193000000000002e-05 2024-09-15T01:32:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288764+0000", 2024-09-15T01:32:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0555999999999997e-05 2024-09-15T01:32:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:96 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.395631+0000", 2024-09-15T01:32:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.089486710999999, 2024-09-15T01:32:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040472160000000002, 2024-09-15T01:32:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 96 2024-09-15T01:32:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395631+0000", 2024-09-15T01:32:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395631+0000", 2024-09-15T01:32:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395631+0000", 2024-09-15T01:32:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4999999999999999e-07 2024-09-15T01:32:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395674+0000", 2024-09-15T01:32:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3263999999999998e-05 2024-09-15T01:32:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395679+0000", 2024-09-15T01:32:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.045e-06 2024-09-15T01:32:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395683+0000", 2024-09-15T01:32:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.811e-06 2024-09-15T01:32:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.398542+0000", 2024-09-15T01:32:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028589700000000002 2024-09-15T01:32:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.398618+0000", 2024-09-15T01:32:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6326000000000001e-05 2024-09-15T01:32:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.398739+0000", 2024-09-15T01:32:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000120847 2024-09-15T01:32:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.398910+0000", 2024-09-15T01:32:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017120500000000001 2024-09-15T01:32:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.398945+0000", 2024-09-15T01:32:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4521999999999999e-05 2024-09-15T01:32:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.398996+0000", 2024-09-15T01:32:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1685999999999999e-05 2024-09-15T01:32:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399238+0000", 2024-09-15T01:32:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024197199999999999 2024-09-15T01:32:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399637+0000", 2024-09-15T01:32:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000398398 2024-09-15T01:32:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399659+0000", 2024-09-15T01:32:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1749000000000001e-05 2024-09-15T01:32:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399678+0000", 2024-09-15T01:32:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9171e-05 2024-09-15T01:32:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:97 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.395687+0000", 2024-09-15T01:32:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.089430297, 2024-09-15T01:32:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048233190000000004, 2024-09-15T01:32:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 97 2024-09-15T01:32:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395687+0000", 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395687+0000", 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395687+0000", 2024-09-15T01:32:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2300000000000002e-07 2024-09-15T01:32:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395689+0000", 2024-09-15T01:32:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0260000000000002e-06 2024-09-15T01:32:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395697+0000", 2024-09-15T01:32:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3019999999999997e-06 2024-09-15T01:32:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395699+0000", 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3120000000000001e-06 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399262+0000", 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003563386 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399338+0000", 2024-09-15T01:32:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5235999999999996e-05 2024-09-15T01:32:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399450+0000", 2024-09-15T01:32:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112132 2024-09-15T01:32:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399588+0000", 2024-09-15T01:32:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013833500000000001 2024-09-15T01:32:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399594+0000", 2024-09-15T01:32:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3199999999999996e-06 2024-09-15T01:32:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399620+0000", 2024-09-15T01:32:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5242999999999999e-05 2024-09-15T01:32:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400056+0000", 2024-09-15T01:32:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000435823 2024-09-15T01:32:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400466+0000", 2024-09-15T01:32:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041044399999999999 2024-09-15T01:32:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400493+0000", 2024-09-15T01:32:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6635000000000002e-05 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400510+0000", 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7702000000000001e-05 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:98 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.395702+0000", 2024-09-15T01:32:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.089415723999998, 2024-09-15T01:32:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048734149999999999, 2024-09-15T01:32:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 98 2024-09-15T01:32:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395702+0000", 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395702+0000", 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395702+0000", 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2999999999999999e-07 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395704+0000", 2024-09-15T01:32:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8160000000000001e-06 2024-09-15T01:32:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395707+0000", 2024-09-15T01:32:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.084e-06 2024-09-15T01:32:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395709+0000", 2024-09-15T01:32:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-09-15T01:32:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399715+0000", 2024-09-15T01:32:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040063989999999999 2024-09-15T01:32:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399789+0000", 2024-09-15T01:32:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4049999999999997e-05 2024-09-15T01:32:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.399904+0000", 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115067 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400034+0000", 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012927699999999999 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400040+0000", 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3210000000000001e-06 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400062+0000", 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1651999999999999e-05 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400523+0000", 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046132999999999998 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400539+0000", 2024-09-15T01:32:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6039999999999999e-05 2024-09-15T01:32:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400558+0000", 2024-09-15T01:32:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9500999999999998e-05 2024-09-15T01:32:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400575+0000", 2024-09-15T01:32:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6487999999999999e-05 2024-09-15T01:32:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:99 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.395716+0000", 2024-09-15T01:32:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.089401539000001, 2024-09-15T01:32:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052892579999999998, 2024-09-15T01:32:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 99 2024-09-15T01:32:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395716+0000", 2024-09-15T01:32:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395716+0000", 2024-09-15T01:32:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395716+0000", 2024-09-15T01:32:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5699999999999999e-07 2024-09-15T01:32:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395747+0000", 2024-09-15T01:32:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1254999999999998e-05 2024-09-15T01:32:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395751+0000", 2024-09-15T01:32:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9430000000000002e-06 2024-09-15T01:32:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395753+0000", 2024-09-15T01:32:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2960000000000001e-06 2024-09-15T01:32:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400073+0000", 2024-09-15T01:32:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004319097 2024-09-15T01:32:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400142+0000", 2024-09-15T01:32:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9507999999999999e-05 2024-09-15T01:32:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400266+0000", 2024-09-15T01:32:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124287 2024-09-15T01:32:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400411+0000", 2024-09-15T01:32:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014435199999999999 2024-09-15T01:32:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400416+0000", 2024-09-15T01:32:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5260000000000001e-06 2024-09-15T01:32:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400463+0000", 2024-09-15T01:32:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6456999999999999e-05 2024-09-15T01:32:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400586+0000", 2024-09-15T01:32:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012369400000000001 2024-09-15T01:32:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400961+0000", 2024-09-15T01:32:33.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037495999999999997 2024-09-15T01:32:33.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400983+0000", 2024-09-15T01:32:33.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1437000000000002e-05 2024-09-15T01:32:33.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401005+0000", 2024-09-15T01:32:33.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2189e-05 2024-09-15T01:32:33.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.634 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.634 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:100 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:33.634 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.395756+0000", 2024-09-15T01:32:33.634 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.089360978999999, 2024-09-15T01:32:33.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0055617000000000002, 2024-09-15T01:32:33.635 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.635 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.635 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 100 2024-09-15T01:32:33.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.635 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395756+0000", 2024-09-15T01:32:33.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395756+0000", 2024-09-15T01:32:33.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395757+0000", 2024-09-15T01:32:33.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:32:33.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395759+0000", 2024-09-15T01:32:33.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2309999999999999e-06 2024-09-15T01:32:33.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395762+0000", 2024-09-15T01:32:33.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9500000000000001e-06 2024-09-15T01:32:33.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395764+0000", 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9580000000000001e-06 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400609+0000", 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048449870000000003 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400678+0000", 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9764999999999994e-05 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400806+0000", 2024-09-15T01:32:33.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012723899999999999 2024-09-15T01:32:33.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400928+0000", 2024-09-15T01:32:33.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012266800000000001 2024-09-15T01:32:33.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400935+0000", 2024-09-15T01:32:33.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4570000000000002e-06 2024-09-15T01:32:33.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400962+0000", 2024-09-15T01:32:33.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7223e-05 2024-09-15T01:32:33.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401017+0000", 2024-09-15T01:32:33.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4868999999999999e-05 2024-09-15T01:32:33.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401246+0000", 2024-09-15T01:32:33.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022941300000000001 2024-09-15T01:32:33.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401287+0000", 2024-09-15T01:32:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0193e-05 2024-09-15T01:32:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401318+0000", 2024-09-15T01:32:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1452000000000003e-05 2024-09-15T01:32:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:124 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-15T01:32:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:18.009681+0000", 2024-09-15T01:32:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.475436755, 2024-09-15T01:32:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003864358, 2024-09-15T01:32:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.644 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 124 2024-09-15T01:32:33.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.644 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.009681+0000", 2024-09-15T01:32:33.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.009681+0000", 2024-09-15T01:32:33.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.009681+0000", 2024-09-15T01:32:33.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:32:33.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.009685+0000", 2024-09-15T01:32:33.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8260000000000003e-06 2024-09-15T01:32:33.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.009691+0000", 2024-09-15T01:32:33.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9179999999999999e-06 2024-09-15T01:32:33.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.009695+0000", 2024-09-15T01:32:33.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.527e-06 2024-09-15T01:32:33.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.012952+0000", 2024-09-15T01:32:33.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032564569999999999 2024-09-15T01:32:33.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.013014+0000", 2024-09-15T01:32:33.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1791000000000005e-05 2024-09-15T01:32:33.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.013093+0000", 2024-09-15T01:32:33.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9116000000000006e-05 2024-09-15T01:32:33.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.013199+0000", 2024-09-15T01:32:33.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010671300000000001 2024-09-15T01:32:33.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.013207+0000", 2024-09-15T01:32:33.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.311e-06 2024-09-15T01:32:33.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.013232+0000", 2024-09-15T01:32:33.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5616e-05 2024-09-15T01:32:33.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.013303+0000", 2024-09-15T01:32:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0822999999999996e-05 2024-09-15T01:32:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.013504+0000", 2024-09-15T01:32:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020097999999999999 2024-09-15T01:32:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.013526+0000", 2024-09-15T01:32:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2181e-05 2024-09-15T01:32:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:18.013545+0000", 2024-09-15T01:32:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8479e-05 2024-09-15T01:32:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:145 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~28] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-15T01:32:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:19.474868+0000", 2024-09-15T01:32:33.652 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.010249186999999, 2024-09-15T01:32:33.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071687239999999996, 2024-09-15T01:32:33.652 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:33.652 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:33.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:33.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:33.652 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145 2024-09-15T01:32:33.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.652 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:33.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:33.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474868+0000", 2024-09-15T01:32:33.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:33.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474868+0000", 2024-09-15T01:32:33.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:33.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:33.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474869+0000", 2024-09-15T01:32:33.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3399999999999999e-07 2024-09-15T01:32:33.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:33.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474873+0000", 2024-09-15T01:32:33.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-09-15T01:32:33.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:33.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474883+0000", 2024-09-15T01:32:33.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0115000000000001e-05 2024-09-15T01:32:33.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474887+0000", 2024-09-15T01:32:33.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7000000000000002e-06 2024-09-15T01:32:33.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:33.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474910+0000", 2024-09-15T01:32:33.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099e-05 2024-09-15T01:32:33.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:33.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474961+0000", 2024-09-15T01:32:33.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1285000000000001e-05 2024-09-15T01:32:33.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:33.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475027+0000", 2024-09-15T01:32:33.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5427999999999998e-05 2024-09-15T01:32:33.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475105+0000", 2024-09-15T01:32:33.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8459000000000002e-05 2024-09-15T01:32:33.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475133+0000", 2024-09-15T01:32:33.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8371999999999999e-05 2024-09-15T01:32:33.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475163+0000", 2024-09-15T01:32:33.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9181999999999999e-05 2024-09-15T01:32:33.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:33.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475179+0000", 2024-09-15T01:32:33.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6069000000000001e-05 2024-09-15T01:32:33.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:33.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.481979+0000", 2024-09-15T01:32:33.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00680038 2024-09-15T01:32:33.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.482007+0000", 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7648e-05 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.482037+0000", 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0048000000000001e-05 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:33.660 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:33.660 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:33.661 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:32:33.662 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:33.658+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:33.762 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:33.760+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:33.780 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:32:33.862 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:33.860+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:33.877 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:32:33.877 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:32:33.877 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:33.885 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:32:33.963 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:33.961+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:33.985 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:32:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:32:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:32:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:8 1.0 e23/22 1:30306672:devicehealth::main.db.0000000000000000:head v 23'3, mlcod=23'1)", 2024-09-15T01:32:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:16.756840+0000", 2024-09-15T01:32:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 17.225096369999999, 2024-09-15T01:32:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00054759899999999996, 2024-09-15T01:32:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.756840+0000", 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.756840+0000", 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.756842+0000", 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6700000000000001e-06 2024-09-15T01:32:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.756854+0000", 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.261e-05 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.756921+0000", 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7036e-05 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.756933+0000", 2024-09-15T01:32:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1534e-05 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.757013+0000", 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0625999999999997e-05 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.757048+0000", 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4662000000000001e-05 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.757165+0000", 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116617 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.757175+0000", 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8479999999999996e-06 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.757212+0000", 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7728000000000002e-05 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.757265+0000", 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2991000000000001e-05 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.757388+0000", 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012227700000000001 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:20 1.0 e23/22 1:30306672:devicehealth::main.db.0000000000000000:head v 23'10, mlcod=23'8)", 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:16.767932+0000", 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 17.214004206999999, 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061749000000000005, 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.767932+0000", 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.767932+0000", 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.767936+0000", 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1529999999999999e-06 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768078+0000", 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014130699999999999 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768103+0000", 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5388000000000002e-05 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768108+0000", 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3920000000000002e-06 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768152+0000", 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4070999999999999e-05 2024-09-15T01:32:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768170+0000", 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7578999999999999e-05 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768291+0000", 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120893 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768304+0000", 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.269e-05 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768372+0000", 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8905999999999994e-05 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768395+0000", 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2039e-05 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768550+0000", 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000155072 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:21 1.0 e23/22 1:30306672:devicehealth::main.db.0000000000000000:head v 23'11, mlcod=23'8)", 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:16.768475+0000", 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 17.213460925, 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00063790699999999995, 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768475+0000", 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768475+0000", 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768476+0000", 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9100000000000004e-07 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768620+0000", 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014361899999999999 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768647+0000", 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7501999999999999e-05 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768652+0000", 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4290000000000003e-06 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768688+0000", 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6279999999999998e-05 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768709+0000", 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1591999999999999e-05 2024-09-15T01:32:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768850+0000", 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014067499999999999 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768859+0000", 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.867e-06 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768923+0000", 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4016000000000005e-05 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.768944+0000", 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0618000000000001e-05 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:16.769113+0000", 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016971799999999999 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:39 1.0 e23/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 23'21, mlcod=23'19)", 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.145825+0000", 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.836111070000001, 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00059400800000000004, 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.145825+0000", 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.145825+0000", 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.145827+0000", 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7099999999999999e-06 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.145839+0000", 2024-09-15T01:32:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1960000000000001e-05 2024-09-15T01:32:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.145908+0000", 2024-09-15T01:32:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9245999999999994e-05 2024-09-15T01:32:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.145919+0000", 2024-09-15T01:32:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1182999999999999e-05 2024-09-15T01:32:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146041+0000", 2024-09-15T01:32:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000121209 2024-09-15T01:32:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146084+0000", 2024-09-15T01:32:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3312999999999997e-05 2024-09-15T01:32:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146172+0000", 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7892000000000005e-05 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146185+0000", 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3413e-05 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146225+0000", 2024-09-15T01:32:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9774999999999998e-05 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146306+0000", 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1063000000000006e-05 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146419+0000", 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000113244 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:40 1.0 e23/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 23'22, mlcod=23'19)", 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.145989+0000", 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.835947028, 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057447499999999996, 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.145989+0000", 2024-09-15T01:32:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.145989+0000", 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.145990+0000", 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7300000000000005e-07 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146003+0000", 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2587e-05 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146056+0000", 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2896000000000003e-05 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146064+0000", 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9529999999999999e-06 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146199+0000", 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000135888 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146215+0000", 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5335000000000001e-05 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146291+0000", 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6415000000000006e-05 2024-09-15T01:32:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146297+0000", 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9190000000000004e-06 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146322+0000", 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4355999999999999e-05 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146371+0000", 2024-09-15T01:32:34.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9594e-05 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146564+0000", 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000192659 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:41 1.0 e23/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 23'23, mlcod=23'19)", 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.146346+0000", 2024-09-15T01:32:34.013 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.835590439000001, 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057837199999999996, 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146346+0000", 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146346+0000", 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146346+0000", 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6100000000000001e-07 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146433+0000", 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6518000000000001e-05 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146460+0000", 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6973000000000001e-05 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146470+0000", 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9669999999999996e-06 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146505+0000", 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4790999999999999e-05 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146517+0000", 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1851000000000001e-05 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146612+0000", 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5402999999999994e-05 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146620+0000", 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0839999999999992e-06 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146676+0000", 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5597000000000003e-05 2024-09-15T01:32:34.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146776+0000", 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000100313 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.146924+0000", 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000148314 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:58 1.0 e23/22 1:30306672:devicehealth::main.db.0000000000000000:head v 23'40, mlcod=23'38)", 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.154877+0000", 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.827059218999999, 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053066400000000003, 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.154877+0000", 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.154877+0000", 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.154879+0000", 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.86e-06 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.154914+0000", 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4699000000000001e-05 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.154982+0000", 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8269000000000007e-05 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.154993+0000", 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1224e-05 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.155094+0000", 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010035900000000001 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.155117+0000", 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3482e-05 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.155216+0000", 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9263000000000002e-05 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.155228+0000", 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2114e-05 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.155266+0000", 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7910999999999997e-05 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.155309+0000", 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2509999999999998e-05 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.155408+0000", 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8973000000000003e-05 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.023 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:75 1.0 e23/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 23'53, mlcod=23'48)", 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.279064+0000", 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.702872504999998, 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00060687399999999998, 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279064+0000", 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279064+0000", 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279064+0000", 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0400000000000002e-07 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279300+0000", 2024-09-15T01:32:34.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023570899999999999 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279335+0000", 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5438000000000002e-05 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279341+0000", 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279372+0000", 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.129e-05 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279386+0000", 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4195e-05 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279469+0000", 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3131000000000001e-05 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279477+0000", 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1179999999999992e-06 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279533+0000", 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5653999999999999e-05 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279588+0000", 2024-09-15T01:32:34.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4446e-05 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.279671+0000", 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3159999999999997e-05 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:77 1.0 e23/22 1:30306672:devicehealth::main.db.0000000000000000:head v 23'55, mlcod=23'53)", 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.280338+0000", 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.701598355000002, 2024-09-15T01:32:34.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00055544199999999996, 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280338+0000", 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280338+0000", 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280338+0000", 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7e-07 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280456+0000", 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117781 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280478+0000", 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1668999999999999e-05 2024-09-15T01:32:34.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280483+0000", 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2580000000000003e-06 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280511+0000", 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8209999999999999e-05 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280523+0000", 2024-09-15T01:32:34.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1429e-05 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280600+0000", 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7510999999999997e-05 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280610+0000", 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0108999999999999e-05 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280694+0000", 2024-09-15T01:32:34.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3907999999999997e-05 2024-09-15T01:32:34.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280720+0000", 2024-09-15T01:32:34.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5262e-05 2024-09-15T01:32:34.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280893+0000", 2024-09-15T01:32:34.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017383500000000001 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:78 1.0 e23/22 1:30306672:devicehealth::main.db.0000000000000000:head v 23'56, mlcod=23'53)", 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.280668+0000", 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.701268523, 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052816500000000004, 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280668+0000", 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280668+0000", 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280668+0000", 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8999999999999996e-07 2024-09-15T01:32:34.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280728+0000", 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9800999999999999e-05 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280769+0000", 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0568000000000002e-05 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280774+0000", 2024-09-15T01:32:34.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3460000000000001e-06 2024-09-15T01:32:34.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280824+0000", 2024-09-15T01:32:34.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9404000000000003e-05 2024-09-15T01:32:34.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280847+0000", 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3691999999999999e-05 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280977+0000", 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000129245 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.280986+0000", 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0950000000000002e-06 2024-09-15T01:32:34.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281053+0000", 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7347000000000004e-05 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281071+0000", 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8073000000000001e-05 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281196+0000", 2024-09-15T01:32:34.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012490400000000001 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:79 1.0 e23/22 1:30306672:devicehealth::main.db.0000000000000000:head v 23'57, mlcod=23'53)", 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.281015+0000", 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.700921615999999, 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053006400000000001, 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281015+0000", 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281015+0000", 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281022+0000", 2024-09-15T01:32:34.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1640000000000004e-06 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281085+0000", 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2654e-05 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281168+0000", 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3863000000000007e-05 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281173+0000", 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2400000000000001e-06 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281203+0000", 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0029999999999999e-05 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281221+0000", 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8478000000000002e-05 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281336+0000", 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000114683 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281345+0000", 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7050000000000005e-06 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281404+0000", 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9490999999999997e-05 2024-09-15T01:32:34.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281422+0000", 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7623000000000001e-05 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.281545+0000", 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012313299999999999 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:80 1.0 e23/22 1:30306672:devicehealth::main.db.0000000000000000:head v 23'58, mlcod=23'53)", 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.288096+0000", 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.693839958000002, 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00059471600000000002, 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288096+0000", 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288096+0000", 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288097+0000", 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2099999999999995e-07 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288286+0000", 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000188965 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288312+0000", 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5687e-05 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288317+0000", 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8640000000000004e-06 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288350+0000", 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3349999999999997e-05 2024-09-15T01:32:34.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288364+0000", 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3694e-05 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288456+0000", 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2447000000000004e-05 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288491+0000", 2024-09-15T01:32:34.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4891000000000002e-05 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288554+0000", 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2979999999999997e-05 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288566+0000", 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2275e-05 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.288691+0000", 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012474200000000001 2024-09-15T01:32:34.052 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:83 1.0 e23/22 1:30306672:devicehealth::main.db.0000000000000000:head v 23'61, mlcod=23'59)", 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.391863+0000", 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.590073493999999, 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00059400399999999997, 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.391863+0000", 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.391863+0000", 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.391866+0000", 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7209999999999999e-06 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.391885+0000", 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9157e-05 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.391931+0000", 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6529999999999997e-05 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.391946+0000", 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4657999999999999e-05 2024-09-15T01:32:34.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.392003+0000", 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7033000000000001e-05 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.392025+0000", 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1951999999999999e-05 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.392137+0000", 2024-09-15T01:32:34.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011239999999999999 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.392173+0000", 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6146000000000003e-05 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.392239+0000", 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5475999999999994e-05 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.392287+0000", 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8541999999999997e-05 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.392457+0000", 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016938899999999999 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:88 1.0 e23/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 23'62, mlcod=23'60)", 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.395649+0000", 2024-09-15T01:32:34.058 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.586287195000001, 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00068026399999999998, 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395649+0000", 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395649+0000", 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395651+0000", 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5659999999999999e-06 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395667+0000", 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5930000000000002e-05 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395730+0000", 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3323000000000003e-05 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395741+0000", 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.096e-05 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395912+0000", 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017132099999999999 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395960+0000", 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7973000000000003e-05 2024-09-15T01:32:34.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.396105+0000", 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014448400000000001 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.396134+0000", 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9459999999999999e-05 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.396167+0000", 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2648999999999999e-05 2024-09-15T01:32:34.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.396260+0000", 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3368000000000005e-05 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.396329+0000", 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.923e-05 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:89 1.0 e23/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 23'63, mlcod=23'60)", 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.395872+0000", 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.586064521000001, 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061156299999999997, 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395872+0000", 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395872+0000", 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395873+0000", 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.22e-06 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395902+0000", 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8529000000000001e-05 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395955+0000", 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3440000000000003e-05 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.395965+0000", 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0247e-05 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.396126+0000", 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016079899999999999 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.396140+0000", 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3961999999999999e-05 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.396247+0000", 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010741199999999999 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.396261+0000", 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3284e-05 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.396295+0000", 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3809000000000001e-05 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.396455+0000", 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016036100000000001 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.396483+0000", 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8500000000000002e-05 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.068 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:101 1.0 e24/22 1:30306672:devicehealth::main.db.0000000000000000:head v 24'75, mlcod=23'73)", 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.903663+0000", 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.078273661000001, 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00087328700000000004, 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903663+0000", 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903663+0000", 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903663+0000", 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.22e-07 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903729+0000", 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6029000000000001e-05 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903753+0000", 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.41e-05 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903781+0000", 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7970000000000002e-05 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903862+0000", 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0807000000000001e-05 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903896+0000", 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3788999999999998e-05 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904078+0000", 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000182623 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904119+0000", 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0898e-05 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904196+0000", 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7080000000000001e-05 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904496+0000", 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00029937799999999999 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904536+0000", 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0391e-05 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:102 1.0 e24/22 1:30306672:devicehealth::main.db.0000000000000000:head v 24'76, mlcod=23'73)", 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.903832+0000", 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.078104557, 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000740083, 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.074 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903832+0000", 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903832+0000", 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903832+0000", 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2300000000000001e-07 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903895+0000", 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2315000000000002e-05 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903925+0000", 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0760999999999999e-05 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903936+0000", 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0387999999999999e-05 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904115+0000", 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017945 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904135+0000", 2024-09-15T01:32:34.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9908e-05 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904301+0000", 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000165666 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904327+0000", 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5922e-05 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904394+0000", 2024-09-15T01:32:34.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7447999999999995e-05 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904544+0000", 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000149269 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904572+0000", 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8333000000000001e-05 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:103 1.0 e24/22 1:30306672:devicehealth::main.db.0000000000000000:head v 24'77, mlcod=23'73)", 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.904064+0000", 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.077871836, 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000633324, 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904064+0000", 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904064+0000", 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904065+0000", 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1200000000000002e-07 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904127+0000", 2024-09-15T01:32:34.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2133999999999998e-05 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904164+0000", 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7073000000000002e-05 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904170+0000", 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8869999999999997e-06 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904362+0000", 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019189399999999999 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904376+0000", 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3542e-05 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904480+0000", 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000104255 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904486+0000", 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.319e-06 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904547+0000", 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0495000000000003e-05 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904575+0000", 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8495000000000001e-05 2024-09-15T01:32:34.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904698+0000", 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012251800000000001 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:104 1.0 e24/22 1:30306672:devicehealth::main.db.0000000000000000:head v 24'78, mlcod=23'73)", 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.904306+0000", 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.077630186, 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00070436499999999998, 2024-09-15T01:32:34.085 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904306+0000", 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904306+0000", 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904307+0000", 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.178e-06 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904402+0000", 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5169999999999999e-05 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904446+0000", 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3041999999999999e-05 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904452+0000", 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9689999999999997e-06 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904699+0000", 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00024607999999999997 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904710+0000", 2024-09-15T01:32:34.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1138000000000001e-05 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904808+0000", 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8516000000000003e-05 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904818+0000", 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7399999999999999e-06 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904877+0000", 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9414999999999999e-05 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904891+0000", 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3370000000000001e-05 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.905010+0000", 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119747 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.090 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:126 1.0 e24/22 1:30306672:devicehealth::main.db.0000000000000000:head v 24'96, mlcod=24'93)", 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:18.014300+0000", 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 15.967636325000001, 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00064430499999999996, 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014300+0000", 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:34.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014300+0000", 2024-09-15T01:32:34.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:34.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:34.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014300+0000", 2024-09-15T01:32:34.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5200000000000002e-07 2024-09-15T01:32:34.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014390+0000", 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0023000000000002e-05 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014426+0000", 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5462e-05 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014432+0000", 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.604e-06 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014465+0000", 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3345e-05 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014476+0000", 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1464000000000001e-05 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014644+0000", 2024-09-15T01:32:34.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016748199999999999 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014654+0000", 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9729999999999992e-06 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014704+0000", 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0661000000000002e-05 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014893+0000", 2024-09-15T01:32:34.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018813900000000001 2024-09-15T01:32:34.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:34.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:34.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:34.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:18.014944+0000", 2024-09-15T01:32:34.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1700000000000003e-05 2024-09-15T01:32:34.096 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.096 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.096 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.096 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:34.096 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:34.096 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:34.098 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:32:34.098 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:34.096+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:34.163 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-15T01:32:34.164 INFO:teuthology.orchestra.run.smithi042.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-15T01:32:34.198 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:34.196+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:34.244 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:34.244 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:34.244 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:32:34.244 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:34.252 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:32:34.274 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:32:34.296 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:32:34.296 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:32:34.299 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:34.298+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:34.348 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:34.348 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:34.348 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:32:34.348 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:32:34.349 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:34.357 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:32:34.400 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:34.398+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:34.453 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:34.453 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:32:34.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:32:34.453 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2024-09-15T01:32:34.453 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:34.461 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:32:34.501 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:34.499+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:34.560 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:34.560 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:34.560 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:32:34.560 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:34.568 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:32:34.602 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:34.600+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:34.664 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:34.665 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:34.665 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:32:34.665 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:32:34.665 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:34.672 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:32:34.702 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:34.701+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:34.769 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:34.770 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:32:34.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:32:34.770 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2024-09-15T01:32:34.770 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:34.778 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-15T01:32:34.804 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:34.802+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:34.876 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:32:34.877 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:34.884 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-15T01:32:34.905 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:34.903+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:34.980 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:34.980 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:34.980 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:32:34.980 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:32:34.980 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:34.988 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-15T01:32:35.032 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:35.031+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:35.084 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:35.084 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:32:35.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:32:35.084 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2024-09-15T01:32:35.085 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:35.092 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:32:35.133 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:35.132+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:35.192 DEBUG:teuthology.orchestra.run.smithi175:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-09-15T01:32:35.202 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:32:35.202 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:32:35.202 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:32:35.202 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:32:35.207 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:35.206+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:35.211 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout:adding op weight read -> 100 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout:adding op weight delete -> 50 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout:adding op weight snap_create -> 50 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout:adding op weight snap_remove -> 50 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout:adding op weight rollback -> 50 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout:adding op weight setattr -> 25 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout:adding op weight rmattr -> 25 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout:adding op weight copy_from -> 50 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout:adding op weight append -> 100 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout:ceph version 17.2.7-1643-gd75f2382 (d75f23820806da622f561ee5ee7551015285993f) quincy (stable) 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout:Configuration: 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout: Number of operations: 400 2024-09-15T01:32:35.217 INFO:tasks.rados.rados.0.smithi175.stdout: Number of objects: 50 2024-09-15T01:32:35.218 INFO:tasks.rados.rados.0.smithi175.stdout: Max in flight operations: 16 2024-09-15T01:32:35.218 INFO:tasks.rados.rados.0.smithi175.stdout: Object size (in bytes): 4000000 2024-09-15T01:32:35.218 INFO:tasks.rados.rados.0.smithi175.stdout: Write stride min: 400000 2024-09-15T01:32:35.218 INFO:tasks.rados.rados.0.smithi175.stdout: Write stride max: 800000 2024-09-15T01:32:35.308 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:35.306+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:35.313 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:32:35.313 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:32:35.313 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:32:35.313 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:32:35.313 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:32:35.321 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:32:35.410 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:35.408+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:35.417 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:32:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:32:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:32:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:32:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4302.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T01:32:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:23.147530+0000", 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.267743964999999, 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0086012669999999992, 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4302", 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.42:0/3794971532", 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 2 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147530+0000", 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147530+0000", 2024-09-15T01:32:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147533+0000", 2024-09-15T01:32:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.587e-06 2024-09-15T01:32:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147545+0000", 2024-09-15T01:32:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1912999999999999e-05 2024-09-15T01:32:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147569+0000", 2024-09-15T01:32:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4389e-05 2024-09-15T01:32:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147583+0000", 2024-09-15T01:32:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3913e-05 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147699+0000", 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116353 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147705+0000", 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7289999999999997e-06 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.155973+0000", 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0082681109999999999 2024-09-15T01:32:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.156131+0000", 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015827199999999999 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4302.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:23.156392+0000", 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.258882523, 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000144408, 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4302", 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.42:0/3794971532", 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.156392+0000", 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.156392+0000", 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.156393+0000", 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0410000000000001e-06 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.156399+0000", 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.088e-06 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.156410+0000", 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1807000000000001e-05 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.156416+0000", 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7690000000000002e-06 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.156447+0000", 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1189999999999998e-05 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.156536+0000", 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8512999999999998e-05 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4302.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:24.136876+0000", 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.278398167000001, 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019791909999999999, 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:32:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4302", 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.42:0/3794971532", 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.136876+0000", 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.136876+0000", 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.136877+0000", 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2899999999999999e-06 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.136885+0000", 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.447e-06 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.136898+0000", 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3895e-05 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.136904+0000", 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5099999999999998e-06 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.136964+0000", 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9669000000000001e-05 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op must wait for map", 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.136969+0000", 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9690000000000002e-06 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.137765+0000", 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00079681399999999996 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.137875+0000", 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109887 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 12", 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.137997+0000", 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138153+0000", 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015612800000000001 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138162+0000", 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1510000000000004e-06 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138193+0000", 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1278000000000001e-05 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138248+0000", 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5549e-05 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138798+0000", 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00054916600000000002 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138830+0000", 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2574999999999999e-05 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138855+0000", 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4862999999999999e-05 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.4302.0:5 2.2 e26/25 ondisk, result = 0)", 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:24.138691+0000", 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.276583492, 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016994199999999999, 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:32:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138691+0000", 2024-09-15T01:32:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138691+0000", 2024-09-15T01:32:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138691+0000", 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7400000000000002e-07 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138713+0000", 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1554000000000001e-05 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138739+0000", 2024-09-15T01:32:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6004e-05 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138745+0000", 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4160000000000001e-06 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138788+0000", 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2966000000000002e-05 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138793+0000", 2024-09-15T01:32:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9019999999999998e-06 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:24.138860+0000", 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7225999999999996e-05 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4302.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:25.142316+0000", 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.272957946, 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0033308790000000001, 2024-09-15T01:32:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4302", 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.42:0/3794971532", 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 7 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.142316+0000", 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.142316+0000", 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.142319+0000", 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4940000000000002e-06 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.142331+0000", 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2262999999999999e-05 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.142362+0000", 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0666000000000001e-05 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.142381+0000", 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9893000000000001e-05 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.142506+0000", 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012499200000000001 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op must wait for map", 2024-09-15T01:32:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.142526+0000", 2024-09-15T01:32:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9253e-05 2024-09-15T01:32:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.143534+0000", 2024-09-15T01:32:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001008316 2024-09-15T01:32:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:32:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.143602+0000", 2024-09-15T01:32:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8273000000000002e-05 2024-09-15T01:32:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 12", 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.143709+0000", 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010647800000000001 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.143895+0000", 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000186517 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.143902+0000", 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9959999999999996e-06 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.143943+0000", 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0930999999999998e-05 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.144150+0000", 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020676200000000001 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.145587+0000", 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014372829999999999 2024-09-15T01:32:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.145621+0000", 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3630000000000002e-05 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.145647+0000", 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6132e-05 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.4302.0:7 2.2 e27/25 ondisk, result = 0)", 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:25.145486+0000", 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.269788512, 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000167553, 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.145486+0000", 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.145486+0000", 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.145487+0000", 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1999999999999999e-06 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.145493+0000", 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5370000000000003e-06 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.145532+0000", 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8852e-05 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.145540+0000", 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0080000000000002e-06 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.145578+0000", 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8210000000000002e-05 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.145584+0000", 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.48e-06 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:25.145653+0000", 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9265999999999997e-05 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:35.448 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:32:35.449 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:32:35.511 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:35.509+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:35.547 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:32:35.547 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:32:35.547 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:32:35.547 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:32:35.556 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:32:35.611 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:35.610+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:35.652 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:32:35.652 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:32:35.652 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:32:35.652 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:32:35.652 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:32:35.660 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:32:35.712 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:35.709+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:35.757 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:32:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:32:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:32:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [] 2024-09-15T01:32:35.757 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:32:35.765 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:32:35.813 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:35.812+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:35.903 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:32:35.903 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:32:35.903 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:32:35.903 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:32:35.911 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:32:35.914 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:35.913+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:36.007 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:32:36.007 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:32:36.008 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:32:36.008 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:32:36.008 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:32:36.015 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:32:36.016 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:36.014+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:36.082 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-15T01:32:36.082 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:32:36.082 INFO:tasks.thrashosds.thrasher:Removing osd 9, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] 2024-09-15T01:32:36.082 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 9 2024-09-15T01:32:36.113 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:32:36.113 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:32:36.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:32:36.113 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [] 2024-09-15T01:32:36.113 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:32:36.116 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:36.115+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:36.122 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-15T01:32:36.171 INFO:tasks.rados.rados.0.smithi175.stdout:1: write initial oid 1 2024-09-15T01:32:36.171 INFO:tasks.rados.rados.0.smithi175.stdout:1: seq_num 0 ranges {0=32768} 2024-09-15T01:32:36.171 INFO:tasks.rados.rados.0.smithi175.stdout:1: writing smithi17529310-1 from 0 to 32768 tid 1 2024-09-15T01:32:36.171 INFO:tasks.rados.rados.0.smithi175.stdout:2: write initial oid 2 2024-09-15T01:32:36.171 INFO:tasks.rados.rados.0.smithi175.stdout:2: seq_num 1 ranges {0=32768} 2024-09-15T01:32:36.171 INFO:tasks.rados.rados.0.smithi175.stdout:2: writing smithi17529310-2 from 0 to 32768 tid 1 2024-09-15T01:32:36.171 INFO:tasks.rados.rados.0.smithi175.stdout:3: write initial oid 3 2024-09-15T01:32:36.172 INFO:tasks.rados.rados.0.smithi175.stdout:3: seq_num 2 ranges {0=49152} 2024-09-15T01:32:36.172 INFO:tasks.rados.rados.0.smithi175.stdout:3: writing smithi17529310-3 from 0 to 49152 tid 1 2024-09-15T01:32:36.172 INFO:tasks.rados.rados.0.smithi175.stdout:4: write initial oid 4 2024-09-15T01:32:36.172 INFO:tasks.rados.rados.0.smithi175.stdout:4: seq_num 3 ranges {0=65536} 2024-09-15T01:32:36.173 INFO:tasks.rados.rados.0.smithi175.stdout:4: writing smithi17529310-4 from 0 to 65536 tid 1 2024-09-15T01:32:36.173 INFO:tasks.rados.rados.0.smithi175.stdout:5: write initial oid 5 2024-09-15T01:32:36.173 INFO:tasks.rados.rados.0.smithi175.stdout:5: seq_num 4 ranges {0=81920} 2024-09-15T01:32:36.173 INFO:tasks.rados.rados.0.smithi175.stdout:5: writing smithi17529310-5 from 0 to 81920 tid 1 2024-09-15T01:32:36.173 INFO:tasks.rados.rados.0.smithi175.stdout:6: write initial oid 6 2024-09-15T01:32:36.173 INFO:tasks.rados.rados.0.smithi175.stdout:6: seq_num 5 ranges {0=98304} 2024-09-15T01:32:36.173 INFO:tasks.rados.rados.0.smithi175.stdout:6: writing smithi17529310-6 from 0 to 98304 tid 1 2024-09-15T01:32:36.173 INFO:tasks.rados.rados.0.smithi175.stdout:7: write initial oid 7 2024-09-15T01:32:36.173 INFO:tasks.rados.rados.0.smithi175.stdout:7: seq_num 6 ranges {0=114688} 2024-09-15T01:32:36.174 INFO:tasks.rados.rados.0.smithi175.stdout:7: writing smithi17529310-7 from 0 to 114688 tid 1 2024-09-15T01:32:36.174 INFO:tasks.rados.rados.0.smithi175.stdout:8: write initial oid 8 2024-09-15T01:32:36.174 INFO:tasks.rados.rados.0.smithi175.stdout:8: seq_num 7 ranges {0=131072} 2024-09-15T01:32:36.174 INFO:tasks.rados.rados.0.smithi175.stdout:8: writing smithi17529310-8 from 0 to 131072 tid 1 2024-09-15T01:32:36.175 INFO:tasks.rados.rados.0.smithi175.stdout:9: write initial oid 9 2024-09-15T01:32:36.175 INFO:tasks.rados.rados.0.smithi175.stdout:9: seq_num 8 ranges {0=147456} 2024-09-15T01:32:36.175 INFO:tasks.rados.rados.0.smithi175.stdout:9: writing smithi17529310-9 from 0 to 147456 tid 1 2024-09-15T01:32:36.175 INFO:tasks.rados.rados.0.smithi175.stdout:10: write initial oid 10 2024-09-15T01:32:36.175 INFO:tasks.rados.rados.0.smithi175.stdout:10: seq_num 9 ranges {0=163840} 2024-09-15T01:32:36.176 INFO:tasks.rados.rados.0.smithi175.stdout:10: writing smithi17529310-10 from 0 to 163840 tid 1 2024-09-15T01:32:36.177 INFO:tasks.rados.rados.0.smithi175.stdout:11: write initial oid 11 2024-09-15T01:32:36.177 INFO:tasks.rados.rados.0.smithi175.stdout:11: seq_num 10 ranges {0=180224} 2024-09-15T01:32:36.178 INFO:tasks.rados.rados.0.smithi175.stdout:11: writing smithi17529310-11 from 0 to 180224 tid 1 2024-09-15T01:32:36.178 INFO:tasks.rados.rados.0.smithi175.stdout:12: write initial oid 12 2024-09-15T01:32:36.178 INFO:tasks.rados.rados.0.smithi175.stdout:12: seq_num 11 ranges {0=196608} 2024-09-15T01:32:36.180 INFO:tasks.rados.rados.0.smithi175.stdout:12: writing smithi17529310-12 from 0 to 196608 tid 1 2024-09-15T01:32:36.180 INFO:tasks.rados.rados.0.smithi175.stdout:13: write initial oid 13 2024-09-15T01:32:36.180 INFO:tasks.rados.rados.0.smithi175.stdout:13: seq_num 12 ranges {0=212992} 2024-09-15T01:32:36.182 INFO:tasks.rados.rados.0.smithi175.stdout:13: writing smithi17529310-13 from 0 to 212992 tid 1 2024-09-15T01:32:36.182 INFO:tasks.rados.rados.0.smithi175.stdout:14: write initial oid 14 2024-09-15T01:32:36.182 INFO:tasks.rados.rados.0.smithi175.stdout:2: finishing write tid 1 to smithi17529310-2 2024-09-15T01:32:36.182 INFO:tasks.rados.rados.0.smithi175.stdout:1: finishing write tid 1 to smithi17529310-1 2024-09-15T01:32:36.182 INFO:tasks.rados.rados.0.smithi175.stdout:1: finishing write tid 2 to smithi17529310-1 2024-09-15T01:32:36.182 INFO:tasks.rados.rados.0.smithi175.stdout:14: seq_num 13 ranges {0=229376} 2024-09-15T01:32:36.184 INFO:tasks.rados.rados.0.smithi175.stdout:14: writing smithi17529310-14 from 0 to 229376 tid 1 2024-09-15T01:32:36.184 INFO:tasks.rados.rados.0.smithi175.stdout:15: write initial oid 15 2024-09-15T01:32:36.184 INFO:tasks.rados.rados.0.smithi175.stdout:15: seq_num 14 ranges {0=245760} 2024-09-15T01:32:36.186 INFO:tasks.rados.rados.0.smithi175.stdout:15: writing smithi17529310-15 from 0 to 245760 tid 1 2024-09-15T01:32:36.186 INFO:tasks.rados.rados.0.smithi175.stdout:16: write initial oid 16 2024-09-15T01:32:36.186 INFO:tasks.rados.rados.0.smithi175.stdout:16: seq_num 15 ranges {0=262144} 2024-09-15T01:32:36.188 INFO:tasks.rados.rados.0.smithi175.stdout:16: writing smithi17529310-16 from 0 to 262144 tid 1 2024-09-15T01:32:36.188 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.188 INFO:tasks.rados.rados.0.smithi175.stdout:2: finishing write tid 2 to smithi17529310-2 2024-09-15T01:32:36.188 INFO:tasks.rados.rados.0.smithi175.stdout:1: finishing write tid 3 to smithi17529310-1 2024-09-15T01:32:36.188 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-15T01:32:36.188 INFO:tasks.rados.rados.0.smithi175.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-09-15T01:32:36.188 INFO:tasks.rados.rados.0.smithi175.stdout:5: finishing write tid 1 to smithi17529310-5 2024-09-15T01:32:36.188 INFO:tasks.rados.rados.0.smithi175.stdout:10: finishing write tid 1 to smithi17529310-10 2024-09-15T01:32:36.188 INFO:tasks.rados.rados.0.smithi175.stdout:3: finishing write tid 1 to smithi17529310-3 2024-09-15T01:32:36.188 INFO:tasks.rados.rados.0.smithi175.stdout:7: finishing write tid 1 to smithi17529310-7 2024-09-15T01:32:36.189 INFO:tasks.rados.rados.0.smithi175.stdout:1: done (15 left) 2024-09-15T01:32:36.189 INFO:tasks.rados.rados.0.smithi175.stdout:17: write initial oid 17 2024-09-15T01:32:36.189 INFO:tasks.rados.rados.0.smithi175.stdout:17: seq_num 16 ranges {0=278528} 2024-09-15T01:32:36.190 INFO:tasks.rados.rados.0.smithi175.stdout:17: writing smithi17529310-17 from 0 to 278528 tid 1 2024-09-15T01:32:36.190 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.190 INFO:tasks.rados.rados.0.smithi175.stdout:4: finishing write tid 1 to smithi17529310-4 2024-09-15T01:32:36.190 INFO:tasks.rados.rados.0.smithi175.stdout:8: finishing write tid 1 to smithi17529310-8 2024-09-15T01:32:36.191 INFO:tasks.rados.rados.0.smithi175.stdout:7: finishing write tid 2 to smithi17529310-7 2024-09-15T01:32:36.191 INFO:tasks.rados.rados.0.smithi175.stdout:6: finishing write tid 1 to smithi17529310-6 2024-09-15T01:32:36.191 INFO:tasks.rados.rados.0.smithi175.stdout:3: finishing write tid 2 to smithi17529310-3 2024-09-15T01:32:36.191 INFO:tasks.rados.rados.0.smithi175.stdout:6: finishing write tid 2 to smithi17529310-6 2024-09-15T01:32:36.191 INFO:tasks.rados.rados.0.smithi175.stdout:8: finishing write tid 2 to smithi17529310-8 2024-09-15T01:32:36.191 INFO:tasks.rados.rados.0.smithi175.stdout:9: finishing write tid 1 to smithi17529310-9 2024-09-15T01:32:36.191 INFO:tasks.rados.rados.0.smithi175.stdout:9: finishing write tid 2 to smithi17529310-9 2024-09-15T01:32:36.191 INFO:tasks.rados.rados.0.smithi175.stdout:15: finishing write tid 1 to smithi17529310-15 2024-09-15T01:32:36.191 INFO:tasks.rados.rados.0.smithi175.stdout:14: finishing write tid 1 to smithi17529310-14 2024-09-15T01:32:36.191 INFO:tasks.rados.rados.0.smithi175.stdout:14: finishing write tid 2 to smithi17529310-14 2024-09-15T01:32:36.192 INFO:tasks.rados.rados.0.smithi175.stdout:6: finishing write tid 3 to smithi17529310-6 2024-09-15T01:32:36.192 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 6 v 2 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-15T01:32:36.192 INFO:tasks.rados.rados.0.smithi175.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-09-15T01:32:36.192 INFO:tasks.rados.rados.0.smithi175.stdout:6: done (15 left) 2024-09-15T01:32:36.192 INFO:tasks.rados.rados.0.smithi175.stdout:18: write initial oid 18 2024-09-15T01:32:36.192 INFO:tasks.rados.rados.0.smithi175.stdout:18: seq_num 17 ranges {0=294912} 2024-09-15T01:32:36.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:32:36.194 INFO:tasks.rados.rados.0.smithi175.stdout:18: writing smithi17529310-18 from 0 to 294912 tid 1 2024-09-15T01:32:36.194 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.194 INFO:tasks.rados.rados.0.smithi175.stdout:7: finishing write tid 3 to smithi17529310-7 2024-09-15T01:32:36.194 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 7 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-15T01:32:36.194 INFO:tasks.rados.rados.0.smithi175.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-09-15T01:32:36.194 INFO:tasks.rados.rados.0.smithi175.stdout:16: finishing write tid 1 to smithi17529310-16 2024-09-15T01:32:36.194 INFO:tasks.rados.rados.0.smithi175.stdout:8: finishing write tid 3 to smithi17529310-8 2024-09-15T01:32:36.194 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 8 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-15T01:32:36.194 INFO:tasks.rados.rados.0.smithi175.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-09-15T01:32:36.194 INFO:tasks.rados.rados.0.smithi175.stdout:9: finishing write tid 3 to smithi17529310-9 2024-09-15T01:32:36.194 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 9 v 4 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-15T01:32:36.195 INFO:tasks.rados.rados.0.smithi175.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-09-15T01:32:36.195 INFO:tasks.rados.rados.0.smithi175.stdout:7: done (15 left) 2024-09-15T01:32:36.195 INFO:tasks.rados.rados.0.smithi175.stdout:8: done (14 left) 2024-09-15T01:32:36.195 INFO:tasks.rados.rados.0.smithi175.stdout:9: done (13 left) 2024-09-15T01:32:36.195 INFO:tasks.rados.rados.0.smithi175.stdout:19: write initial oid 19 2024-09-15T01:32:36.195 INFO:tasks.rados.rados.0.smithi175.stdout:19: seq_num 18 ranges {0=311296} 2024-09-15T01:32:36.196 INFO:tasks.rados.rados.0.smithi175.stdout:19: writing smithi17529310-19 from 0 to 311296 tid 1 2024-09-15T01:32:36.196 INFO:tasks.rados.rados.0.smithi175.stdout:20: write initial oid 20 2024-09-15T01:32:36.196 INFO:tasks.rados.rados.0.smithi175.stdout:20: seq_num 19 ranges {0=327680} 2024-09-15T01:32:36.196 INFO:tasks.rados.rados.0.smithi175.stdout:20: writing smithi17529310-20 from 0 to 327680 tid 1 2024-09-15T01:32:36.197 INFO:tasks.rados.rados.0.smithi175.stdout:21: write initial oid 21 2024-09-15T01:32:36.197 INFO:tasks.rados.rados.0.smithi175.stdout:21: seq_num 20 ranges {0=344064} 2024-09-15T01:32:36.200 INFO:tasks.rados.rados.0.smithi175.stdout:21: writing smithi17529310-21 from 0 to 344064 tid 1 2024-09-15T01:32:36.200 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.200 INFO:tasks.rados.rados.0.smithi175.stdout:14: finishing write tid 3 to smithi17529310-14 2024-09-15T01:32:36.200 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 14 v 6 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-15T01:32:36.200 INFO:tasks.rados.rados.0.smithi175.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-09-15T01:32:36.200 INFO:tasks.rados.rados.0.smithi175.stdout:2: finishing write tid 3 to smithi17529310-2 2024-09-15T01:32:36.200 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-15T01:32:36.200 INFO:tasks.rados.rados.0.smithi175.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-09-15T01:32:36.200 INFO:tasks.rados.rados.0.smithi175.stdout:3: finishing write tid 3 to smithi17529310-3 2024-09-15T01:32:36.200 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 3 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-15T01:32:36.200 INFO:tasks.rados.rados.0.smithi175.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-09-15T01:32:36.201 INFO:tasks.rados.rados.0.smithi175.stdout:16: finishing write tid 2 to smithi17529310-16 2024-09-15T01:32:36.201 INFO:tasks.rados.rados.0.smithi175.stdout:17: finishing write tid 1 to smithi17529310-17 2024-09-15T01:32:36.201 INFO:tasks.rados.rados.0.smithi175.stdout:2: done (15 left) 2024-09-15T01:32:36.201 INFO:tasks.rados.rados.0.smithi175.stdout:3: done (14 left) 2024-09-15T01:32:36.201 INFO:tasks.rados.rados.0.smithi175.stdout:14: done (13 left) 2024-09-15T01:32:36.201 INFO:tasks.rados.rados.0.smithi175.stdout:22: write initial oid 22 2024-09-15T01:32:36.201 INFO:tasks.rados.rados.0.smithi175.stdout:22: seq_num 21 ranges {0=360448} 2024-09-15T01:32:36.202 INFO:tasks.rados.rados.0.smithi175.stdout:22: writing smithi17529310-22 from 0 to 360448 tid 1 2024-09-15T01:32:36.202 INFO:tasks.rados.rados.0.smithi175.stdout:23: write initial oid 23 2024-09-15T01:32:36.202 INFO:tasks.rados.rados.0.smithi175.stdout:23: seq_num 22 ranges {0=376832} 2024-09-15T01:32:36.205 INFO:tasks.rados.rados.0.smithi175.stdout:23: writing smithi17529310-23 from 0 to 376832 tid 1 2024-09-15T01:32:36.205 INFO:tasks.rados.rados.0.smithi175.stdout:24: write initial oid 24 2024-09-15T01:32:36.205 INFO:tasks.rados.rados.0.smithi175.stdout:24: seq_num 23 ranges {0=393216} 2024-09-15T01:32:36.208 INFO:tasks.rados.rados.0.smithi175.stdout:24: writing smithi17529310-24 from 0 to 393216 tid 1 2024-09-15T01:32:36.208 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.208 INFO:tasks.rados.rados.0.smithi175.stdout:12: finishing write tid 1 to smithi17529310-12 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:16: finishing write tid 3 to smithi17529310-16 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 16 v 4 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:12: finishing write tid 2 to smithi17529310-12 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:13: finishing write tid 1 to smithi17529310-13 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:13: finishing write tid 2 to smithi17529310-13 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:12: finishing write tid 3 to smithi17529310-12 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 12 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:13: finishing write tid 3 to smithi17529310-13 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 13 v 2 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:10: finishing write tid 2 to smithi17529310-10 2024-09-15T01:32:36.209 INFO:tasks.rados.rados.0.smithi175.stdout:5: finishing write tid 2 to smithi17529310-5 2024-09-15T01:32:36.210 INFO:tasks.rados.rados.0.smithi175.stdout:11: finishing write tid 1 to smithi17529310-11 2024-09-15T01:32:36.210 INFO:tasks.rados.rados.0.smithi175.stdout:19: finishing write tid 1 to smithi17529310-19 2024-09-15T01:32:36.210 INFO:tasks.rados.rados.0.smithi175.stdout:12: done (15 left) 2024-09-15T01:32:36.210 INFO:tasks.rados.rados.0.smithi175.stdout:13: done (14 left) 2024-09-15T01:32:36.210 INFO:tasks.rados.rados.0.smithi175.stdout:16: done (13 left) 2024-09-15T01:32:36.210 INFO:tasks.rados.rados.0.smithi175.stdout:25: write initial oid 25 2024-09-15T01:32:36.210 INFO:tasks.rados.rados.0.smithi175.stdout:25: seq_num 24 ranges {0=409600} 2024-09-15T01:32:36.210 INFO:tasks.rados.rados.0.smithi175.stdout:25: writing smithi17529310-25 from 0 to 409600 tid 1 2024-09-15T01:32:36.211 INFO:tasks.rados.rados.0.smithi175.stdout:26: write initial oid 26 2024-09-15T01:32:36.211 INFO:tasks.rados.rados.0.smithi175.stdout:26: seq_num 25 ranges {0=425984} 2024-09-15T01:32:36.213 INFO:tasks.rados.rados.0.smithi175.stdout:26: writing smithi17529310-26 from 0 to 425984 tid 1 2024-09-15T01:32:36.213 INFO:tasks.rados.rados.0.smithi175.stdout:27: write initial oid 27 2024-09-15T01:32:36.213 INFO:tasks.rados.rados.0.smithi175.stdout:27: seq_num 26 ranges {0=442368} 2024-09-15T01:32:36.217 INFO:tasks.rados.rados.0.smithi175.stdout:27: writing smithi17529310-27 from 0 to 442368 tid 1 2024-09-15T01:32:36.217 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.217 INFO:tasks.rados.rados.0.smithi175.stdout:19: finishing write tid 2 to smithi17529310-19 2024-09-15T01:32:36.217 INFO:tasks.rados.rados.0.smithi175.stdout:11: finishing write tid 2 to smithi17529310-11 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:20: finishing write tid 1 to smithi17529310-20 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:20: finishing write tid 2 to smithi17529310-20 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:5: finishing write tid 3 to smithi17529310-5 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 5 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:18: finishing write tid 1 to smithi17529310-18 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:10: finishing write tid 3 to smithi17529310-10 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 10 v 2 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:20: finishing write tid 3 to smithi17529310-20 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 20 v 4 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:19: finishing write tid 3 to smithi17529310-19 2024-09-15T01:32:36.218 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 19 v 4 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:11: finishing write tid 3 to smithi17529310-11 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 11 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:18: finishing write tid 2 to smithi17529310-18 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:21: finishing write tid 1 to smithi17529310-21 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:5: done (15 left) 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:10: done (14 left) 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:11: done (13 left) 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:19: done (12 left) 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:20: done (11 left) 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:28: write initial oid 28 2024-09-15T01:32:36.219 INFO:tasks.rados.rados.0.smithi175.stdout:28: seq_num 27 ranges {0=458752} 2024-09-15T01:32:36.220 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:36.214+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:36.220 INFO:tasks.rados.rados.0.smithi175.stdout:28: writing smithi17529310-28 from 0 to 458752 tid 1 2024-09-15T01:32:36.220 INFO:tasks.rados.rados.0.smithi175.stdout:29: write initial oid 29 2024-09-15T01:32:36.220 INFO:tasks.rados.rados.0.smithi175.stdout:29: seq_num 28 ranges {0=475136} 2024-09-15T01:32:36.222 INFO:tasks.rados.rados.0.smithi175.stdout:29: writing smithi17529310-29 from 0 to 475136 tid 1 2024-09-15T01:32:36.222 INFO:tasks.rados.rados.0.smithi175.stdout:30: write initial oid 30 2024-09-15T01:32:36.222 INFO:tasks.rados.rados.0.smithi175.stdout:30: seq_num 29 ranges {0=491520} 2024-09-15T01:32:36.226 INFO:tasks.rados.rados.0.smithi175.stdout:30: writing smithi17529310-30 from 0 to 491520 tid 1 2024-09-15T01:32:36.226 INFO:tasks.rados.rados.0.smithi175.stdout:31: write initial oid 31 2024-09-15T01:32:36.226 INFO:tasks.rados.rados.0.smithi175.stdout:31: seq_num 30 ranges {0=507904} 2024-09-15T01:32:36.229 INFO:tasks.rados.rados.0.smithi175.stdout:31: writing smithi17529310-31 from 0 to 507904 tid 1 2024-09-15T01:32:36.229 INFO:tasks.rados.rados.0.smithi175.stdout:32: write initial oid 32 2024-09-15T01:32:36.229 INFO:tasks.rados.rados.0.smithi175.stdout:32: seq_num 31 ranges {0=524288} 2024-09-15T01:32:36.232 INFO:tasks.rados.rados.0.smithi175.stdout:32: writing smithi17529310-32 from 0 to 524288 tid 1 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:22: finishing write tid 1 to smithi17529310-22 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:21: finishing write tid 2 to smithi17529310-21 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:22: finishing write tid 2 to smithi17529310-22 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:18: finishing write tid 3 to smithi17529310-18 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 18 v 2 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:22: finishing write tid 3 to smithi17529310-22 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 22 v 6 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:18: done (15 left) 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:22: done (14 left) 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:33: write initial oid 33 2024-09-15T01:32:36.233 INFO:tasks.rados.rados.0.smithi175.stdout:33: seq_num 32 ranges {0=540672} 2024-09-15T01:32:36.235 INFO:tasks.rados.rados.0.smithi175.stdout:33: writing smithi17529310-33 from 0 to 540672 tid 1 2024-09-15T01:32:36.235 INFO:tasks.rados.rados.0.smithi175.stdout:34: write initial oid 34 2024-09-15T01:32:36.235 INFO:tasks.rados.rados.0.smithi175.stdout:34: seq_num 33 ranges {0=557056} 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:72 3.5s0 3:af8bec4d:::smithi17529310-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.030491382000000001, 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030516033000000001, 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.34e-07 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207319+0000", 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207326+0000", 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4860000000000002e-06 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207330+0000", 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0690000000000003e-06 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208687+0000", 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001356966 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208760+0000", 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2663000000000005e-05 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210380+0000", 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001620659 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210464+0000", 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3387999999999995e-05 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:32:36.245 INFO:tasks.rados.rados.0.smithi175.stdout:34: writing smithi17529310-34 from 0 to 557056 tid 1 2024-09-15T01:32:36.245 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.245 INFO:tasks.rados.rados.0.smithi175.stdout:4: finishing write tid 2 to smithi17529310-4 2024-09-15T01:32:36.245 INFO:tasks.rados.rados.0.smithi175.stdout:24: finishing write tid 1 to smithi17529310-24 2024-09-15T01:32:36.245 INFO:tasks.rados.rados.0.smithi175.stdout:24: finishing write tid 2 to smithi17529310-24 2024-09-15T01:32:36.245 INFO:tasks.rados.rados.0.smithi175.stdout:4: finishing write tid 3 to smithi17529310-4 2024-09-15T01:32:36.245 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:25: finishing write tid 1 to smithi17529310-25 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:23: finishing write tid 1 to smithi17529310-23 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:23: finishing write tid 2 to smithi17529310-23 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:23: finishing write tid 3 to smithi17529310-23 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 23 v 2 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:26: finishing write tid 1 to smithi17529310-26 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:21: finishing write tid 3 to smithi17529310-21 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 21 v 4 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:27: finishing write tid 1 to smithi17529310-27 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:27: finishing write tid 2 to smithi17529310-27 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:28: finishing write tid 1 to smithi17529310-28 2024-09-15T01:32:36.246 INFO:tasks.rados.rados.0.smithi175.stdout:28: finishing write tid 2 to smithi17529310-28 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:28: finishing write tid 3 to smithi17529310-28 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 28 v 8 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:17: finishing write tid 2 to smithi17529310-17 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:29: finishing write tid 1 to smithi17529310-29 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:29: finishing write tid 2 to smithi17529310-29 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:17: finishing write tid 3 to smithi17529310-17 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 17 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:30: finishing write tid 1 to smithi17529310-30 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:4: done (15 left) 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:17: done (14 left) 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:21: done (13 left) 2024-09-15T01:32:36.247 INFO:tasks.rados.rados.0.smithi175.stdout:23: done (12 left) 2024-09-15T01:32:36.248 INFO:tasks.rados.rados.0.smithi175.stdout:28: done (11 left) 2024-09-15T01:32:36.248 INFO:tasks.rados.rados.0.smithi175.stdout:35: write initial oid 35 2024-09-15T01:32:36.248 INFO:tasks.rados.rados.0.smithi175.stdout:35: seq_num 34 ranges {0=573440} 2024-09-15T01:32:36.248 INFO:tasks.rados.rados.0.smithi175.stdout:35: writing smithi17529310-35 from 0 to 573440 tid 1 2024-09-15T01:32:36.248 INFO:tasks.rados.rados.0.smithi175.stdout:36: write initial oid 36 2024-09-15T01:32:36.248 INFO:tasks.rados.rados.0.smithi175.stdout:36: seq_num 35 ranges {0=589824} 2024-09-15T01:32:36.248 INFO:tasks.rados.rados.0.smithi175.stdout:36: writing smithi17529310-36 from 0 to 589824 tid 1 2024-09-15T01:32:36.248 INFO:tasks.rados.rados.0.smithi175.stdout:37: write initial oid 37 2024-09-15T01:32:36.249 INFO:tasks.rados.rados.0.smithi175.stdout:37: seq_num 36 ranges {0=606208} 2024-09-15T01:32:36.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-15T01:32:36.250 INFO:tasks.rados.rados.0.smithi175.stdout:37: writing smithi17529310-37 from 0 to 606208 tid 1 2024-09-15T01:32:36.251 INFO:tasks.rados.rados.0.smithi175.stdout:38: write initial oid 38 2024-09-15T01:32:36.251 INFO:tasks.rados.rados.0.smithi175.stdout:38: seq_num 37 ranges {0=622592} 2024-09-15T01:32:36.253 INFO:tasks.rados.rados.0.smithi175.stdout:38: writing smithi17529310-38 from 0 to 622592 tid 1 2024-09-15T01:32:36.254 INFO:tasks.rados.rados.0.smithi175.stdout:39: write initial oid 39 2024-09-15T01:32:36.254 INFO:tasks.rados.rados.0.smithi175.stdout:39: seq_num 38 ranges {0=638976} 2024-09-15T01:32:36.258 INFO:tasks.rados.rados.0.smithi175.stdout:39: writing smithi17529310-39 from 0 to 638976 tid 1 2024-09-15T01:32:36.258 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.258 INFO:tasks.rados.rados.0.smithi175.stdout:30: finishing write tid 2 to smithi17529310-30 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:26: finishing write tid 2 to smithi17529310-26 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:31: finishing write tid 1 to smithi17529310-31 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:31: finishing write tid 2 to smithi17529310-31 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:26: finishing write tid 3 to smithi17529310-26 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 26 v 6 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:32: finishing write tid 1 to smithi17529310-32 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:32: finishing write tid 2 to smithi17529310-32 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:32: finishing write tid 3 to smithi17529310-32 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 32 v 4 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:29: finishing write tid 3 to smithi17529310-29 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 29 v 6 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-09-15T01:32:36.259 INFO:tasks.rados.rados.0.smithi175.stdout:33: finishing write tid 1 to smithi17529310-33 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:34: finishing write tid 1 to smithi17529310-34 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:34: finishing write tid 2 to smithi17529310-34 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:34: finishing write tid 3 to smithi17529310-34 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 34 v 2 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:24: finishing write tid 3 to smithi17529310-24 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 24 v 4 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:35: finishing write tid 1 to smithi17529310-35 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:24: done (15 left) 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:26: done (14 left) 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:29: done (13 left) 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:32: done (12 left) 2024-09-15T01:32:36.260 INFO:tasks.rados.rados.0.smithi175.stdout:34: done (11 left) 2024-09-15T01:32:36.261 INFO:tasks.rados.rados.0.smithi175.stdout:40: write initial oid 40 2024-09-15T01:32:36.261 INFO:tasks.rados.rados.0.smithi175.stdout:40: seq_num 39 ranges {0=671744} 2024-09-15T01:32:36.262 INFO:tasks.rados.rados.0.smithi175.stdout:40: writing smithi17529310-40 from 0 to 671744 tid 1 2024-09-15T01:32:36.262 INFO:tasks.rados.rados.0.smithi175.stdout:41: write initial oid 41 2024-09-15T01:32:36.262 INFO:tasks.rados.rados.0.smithi175.stdout:41: seq_num 40 ranges {0=688128} 2024-09-15T01:32:36.267 INFO:tasks.rados.rados.0.smithi175.stdout:41: writing smithi17529310-41 from 0 to 688128 tid 1 2024-09-15T01:32:36.267 INFO:tasks.rados.rados.0.smithi175.stdout:42: write initial oid 42 2024-09-15T01:32:36.267 INFO:tasks.rados.rados.0.smithi175.stdout:42: seq_num 41 ranges {0=704512} 2024-09-15T01:32:36.271 INFO:tasks.rados.rados.0.smithi175.stdout:42: writing smithi17529310-42 from 0 to 704512 tid 1 2024-09-15T01:32:36.271 INFO:tasks.rados.rados.0.smithi175.stdout:43: write initial oid 43 2024-09-15T01:32:36.271 INFO:tasks.rados.rados.0.smithi175.stdout:43: seq_num 42 ranges {0=720896} 2024-09-15T01:32:36.275 INFO:tasks.rados.rados.0.smithi175.stdout:43: writing smithi17529310-43 from 0 to 720896 tid 1 2024-09-15T01:32:36.275 INFO:tasks.rados.rados.0.smithi175.stdout:44: write initial oid 44 2024-09-15T01:32:36.275 INFO:tasks.rados.rados.0.smithi175.stdout:44: seq_num 43 ranges {0=737280} 2024-09-15T01:32:36.280 INFO:tasks.rados.rados.0.smithi175.stdout:44: writing smithi17529310-44 from 0 to 737280 tid 1 2024-09-15T01:32:36.280 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.280 INFO:tasks.rados.rados.0.smithi175.stdout:27: finishing write tid 3 to smithi17529310-27 2024-09-15T01:32:36.280 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 27 v 6 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-15T01:32:36.280 INFO:tasks.rados.rados.0.smithi175.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-09-15T01:32:36.280 INFO:tasks.rados.rados.0.smithi175.stdout:36: finishing write tid 1 to smithi17529310-36 2024-09-15T01:32:36.280 INFO:tasks.rados.rados.0.smithi175.stdout:37: finishing write tid 1 to smithi17529310-37 2024-09-15T01:32:36.280 INFO:tasks.rados.rados.0.smithi175.stdout:27: done (15 left) 2024-09-15T01:32:36.280 INFO:tasks.rados.rados.0.smithi175.stdout:45: write initial oid 45 2024-09-15T01:32:36.280 INFO:tasks.rados.rados.0.smithi175.stdout:45: seq_num 44 ranges {0=753664} 2024-09-15T01:32:36.285 INFO:tasks.rados.rados.0.smithi175.stdout:45: writing smithi17529310-45 from 0 to 753664 tid 1 2024-09-15T01:32:36.285 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.285 INFO:tasks.rados.rados.0.smithi175.stdout:37: finishing write tid 2 to smithi17529310-37 2024-09-15T01:32:36.285 INFO:tasks.rados.rados.0.smithi175.stdout:37: finishing write tid 3 to smithi17529310-37 2024-09-15T01:32:36.285 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 37 v 4 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-15T01:32:36.285 INFO:tasks.rados.rados.0.smithi175.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-09-15T01:32:36.285 INFO:tasks.rados.rados.0.smithi175.stdout:33: finishing write tid 2 to smithi17529310-33 2024-09-15T01:32:36.285 INFO:tasks.rados.rados.0.smithi175.stdout:38: finishing write tid 1 to smithi17529310-38 2024-09-15T01:32:36.285 INFO:tasks.rados.rados.0.smithi175.stdout:37: done (15 left) 2024-09-15T01:32:36.285 INFO:tasks.rados.rados.0.smithi175.stdout:46: write initial oid 46 2024-09-15T01:32:36.285 INFO:tasks.rados.rados.0.smithi175.stdout:46: seq_num 45 ranges {0=770048} 2024-09-15T01:32:36.289 INFO:tasks.rados.rados.0.smithi175.stdout:46: writing smithi17529310-46 from 0 to 770048 tid 1 2024-09-15T01:32:36.289 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.289 INFO:tasks.rados.rados.0.smithi175.stdout:36: finishing write tid 2 to smithi17529310-36 2024-09-15T01:32:36.289 INFO:tasks.rados.rados.0.smithi175.stdout:36: finishing write tid 3 to smithi17529310-36 2024-09-15T01:32:36.289 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 36 v 8 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-15T01:32:36.289 INFO:tasks.rados.rados.0.smithi175.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-09-15T01:32:36.289 INFO:tasks.rados.rados.0.smithi175.stdout:40: finishing write tid 1 to smithi17529310-40 2024-09-15T01:32:36.289 INFO:tasks.rados.rados.0.smithi175.stdout:36: done (15 left) 2024-09-15T01:32:36.289 INFO:tasks.rados.rados.0.smithi175.stdout:47: write initial oid 47 2024-09-15T01:32:36.289 INFO:tasks.rados.rados.0.smithi175.stdout:47: seq_num 46 ranges {0=786432} 2024-09-15T01:32:36.294 INFO:tasks.rados.rados.0.smithi175.stdout:47: writing smithi17529310-47 from 0 to 786432 tid 1 2024-09-15T01:32:36.294 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.294 INFO:tasks.rados.rados.0.smithi175.stdout:40: finishing write tid 2 to smithi17529310-40 2024-09-15T01:32:36.294 INFO:tasks.rados.rados.0.smithi175.stdout:40: finishing write tid 3 to smithi17529310-40 2024-09-15T01:32:36.294 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 40 v 4 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-15T01:32:36.294 INFO:tasks.rados.rados.0.smithi175.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-09-15T01:32:36.294 INFO:tasks.rados.rados.0.smithi175.stdout:41: finishing write tid 1 to smithi17529310-41 2024-09-15T01:32:36.294 INFO:tasks.rados.rados.0.smithi175.stdout:41: finishing write tid 2 to smithi17529310-41 2024-09-15T01:32:36.294 INFO:tasks.rados.rados.0.smithi175.stdout:40: done (15 left) 2024-09-15T01:32:36.294 INFO:tasks.rados.rados.0.smithi175.stdout:48: write initial oid 48 2024-09-15T01:32:36.294 INFO:tasks.rados.rados.0.smithi175.stdout:48: seq_num 47 ranges {0=425984,425984=376832} 2024-09-15T01:32:36.295 INFO:tasks.rados.rados.0.smithi175.stdout:48: writing smithi17529310-48 from 0 to 425984 tid 1 2024-09-15T01:32:36.298 INFO:tasks.rados.rados.0.smithi175.stdout:48: writing smithi17529310-48 from 425984 to 802816 tid 2 2024-09-15T01:32:36.298 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.298 INFO:tasks.rados.rados.0.smithi175.stdout:41: finishing write tid 3 to smithi17529310-41 2024-09-15T01:32:36.299 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 41 v 6 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-15T01:32:36.299 INFO:tasks.rados.rados.0.smithi175.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-09-15T01:32:36.299 INFO:tasks.rados.rados.0.smithi175.stdout:41: done (15 left) 2024-09-15T01:32:36.299 INFO:tasks.rados.rados.0.smithi175.stdout:49: write initial oid 49 2024-09-15T01:32:36.299 INFO:tasks.rados.rados.0.smithi175.stdout:49: seq_num 48 ranges {0=442368,442368=376832} 2024-09-15T01:32:36.301 INFO:tasks.rados.rados.0.smithi175.stdout:49: writing smithi17529310-49 from 0 to 442368 tid 1 2024-09-15T01:32:36.304 INFO:tasks.rados.rados.0.smithi175.stdout:49: writing smithi17529310-49 from 442368 to 819200 tid 2 2024-09-15T01:32:36.304 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 16 2024-09-15T01:32:36.304 INFO:tasks.rados.rados.0.smithi175.stdout:42: finishing write tid 1 to smithi17529310-42 2024-09-15T01:32:36.304 INFO:tasks.rados.rados.0.smithi175.stdout:42: finishing write tid 2 to smithi17529310-42 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:43: finishing write tid 1 to smithi17529310-43 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:25: finishing write tid 2 to smithi17529310-25 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:44: finishing write tid 1 to smithi17529310-44 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:44: finishing write tid 2 to smithi17529310-44 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:25: finishing write tid 3 to smithi17529310-25 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 25 v 4 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:44: finishing write tid 3 to smithi17529310-44 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 44 v 8 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:45: finishing write tid 1 to smithi17529310-45 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:25: done (15 left) 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:44: done (14 left) 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:50: write initial oid 50 2024-09-15T01:32:36.305 INFO:tasks.rados.rados.0.smithi175.stdout:50: seq_num 49 ranges {0=458752,458752=376832} 2024-09-15T01:32:36.306 INFO:tasks.rados.rados.0.smithi175.stdout:50: writing smithi17529310-50 from 0 to 458752 tid 1 2024-09-15T01:32:36.309 INFO:tasks.rados.rados.0.smithi175.stdout:50: writing smithi17529310-50 from 458752 to 835584 tid 2 2024-09-15T01:32:36.309 INFO:tasks.rados.rados.0.smithi175.stdout:51: delete oid 22 current snap is 0 2024-09-15T01:32:36.309 INFO:tasks.rados.rados.0.smithi175.stdout:45: finishing write tid 2 to smithi17529310-45 2024-09-15T01:32:36.309 INFO:tasks.rados.rados.0.smithi175.stdout:45: finishing write tid 3 to smithi17529310-45 2024-09-15T01:32:36.309 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 45 v 4 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-15T01:32:36.309 INFO:tasks.rados.rados.0.smithi175.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-09-15T01:32:36.309 INFO:tasks.rados.rados.0.smithi175.stdout:38: finishing write tid 2 to smithi17529310-38 2024-09-15T01:32:36.309 INFO:tasks.rados.rados.0.smithi175.stdout:46: finishing write tid 1 to smithi17529310-46 2024-09-15T01:32:36.309 INFO:tasks.rados.rados.0.smithi175.stdout:33: finishing write tid 3 to smithi17529310-33 2024-09-15T01:32:36.309 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 33 v 8 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:46: finishing write tid 2 to smithi17529310-46 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:47: finishing write tid 1 to smithi17529310-47 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:46: finishing write tid 3 to smithi17529310-46 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 46 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:48: finishing write tid 1 to smithi17529310-48 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:38: finishing write tid 3 to smithi17529310-38 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 38 v 10 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:48: finishing write tid 2 to smithi17529310-48 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:48: finishing write tid 3 to smithi17529310-48 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:49: finishing write tid 1 to smithi17529310-49 2024-09-15T01:32:36.310 INFO:tasks.rados.rados.0.smithi175.stdout:49: finishing write tid 2 to smithi17529310-49 2024-09-15T01:32:36.311 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:32:36.312 INFO:tasks.rados.rados.0.smithi175.stdout:31: finishing write tid 3 to smithi17529310-31 2024-09-15T01:32:36.312 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 31 v 8 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-15T01:32:36.312 INFO:tasks.rados.rados.0.smithi175.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-09-15T01:32:36.312 INFO:tasks.rados.rados.0.smithi175.stdout:15: finishing write tid 2 to smithi17529310-15 2024-09-15T01:32:36.312 INFO:tasks.rados.rados.0.smithi175.stdout:50: finishing write tid 1 to smithi17529310-50 2024-09-15T01:32:36.312 INFO:tasks.rados.rados.0.smithi175.stdout:31: done (15 left) 2024-09-15T01:32:36.312 INFO:tasks.rados.rados.0.smithi175.stdout:33: done (14 left) 2024-09-15T01:32:36.312 INFO:tasks.rados.rados.0.smithi175.stdout:38: done (13 left) 2024-09-15T01:32:36.312 INFO:tasks.rados.rados.0.smithi175.stdout:45: done (12 left) 2024-09-15T01:32:36.312 INFO:tasks.rados.rados.0.smithi175.stdout:46: done (11 left) 2024-09-15T01:32:36.312 INFO:tasks.rados.rados.0.smithi175.stdout:51: done (10 left) 2024-09-15T01:32:36.313 INFO:tasks.rados.rados.0.smithi175.stdout:52: read oid 16 snap -1 2024-09-15T01:32:36.313 INFO:tasks.rados.rados.0.smithi175.stdout:52: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-15T01:32:36.313 INFO:tasks.rados.rados.0.smithi175.stdout:53: snap_create 2024-09-15T01:32:36.313 INFO:tasks.rados.rados.0.smithi175.stdout:15: finishing write tid 3 to smithi17529310-15 2024-09-15T01:32:36.313 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 15 v 2 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-15T01:32:36.314 INFO:tasks.rados.rados.0.smithi175.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-09-15T01:32:36.315 INFO:tasks.rados.rados.0.smithi175.stdout:52: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-15T01:32:36.356 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:36.355+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:36.358 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:32:36.358 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:32:36.358 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:32:36.358 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:32:36.358 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:32:36.366 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-15T01:32:36.380 INFO:tasks.rados.rados.0.smithi175.stdout:43: finishing write tid 2 to smithi17529310-43 2024-09-15T01:32:36.381 INFO:tasks.rados.rados.0.smithi175.stdout:43: finishing write tid 3 to smithi17529310-43 2024-09-15T01:32:36.381 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 43 v 8 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-15T01:32:36.382 INFO:tasks.rados.rados.0.smithi175.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-09-15T01:32:36.422 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:36.420+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:36.428 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-15T01:32:36.427+0000 7f79afbf1640 -1 mon.a@0(leader).osd e34 definitely_dead 0 2024-09-15T01:32:36.443 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:32:36.443 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:32:36.452 INFO:tasks.rados.rados.0.smithi175.stdout:48: finishing write tid 4 to smithi17529310-48 2024-09-15T01:32:36.452 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 48 v 9 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-15T01:32:36.452 INFO:tasks.rados.rados.0.smithi175.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-09-15T01:32:36.462 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:32:36.463 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:32:36.477 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:32:36.477 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:32:36.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:32:36.478 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:32:36.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.478 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:10 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [append 0~65536,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:36.478 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.175773+0000", 2024-09-15T01:32:36.478 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.298955788, 2024-09-15T01:32:36.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011565234000000001, 2024-09-15T01:32:36.478 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.478 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:36.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:32:36.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:32:36.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 10 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175773+0000", 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175773+0000", 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175775+0000", 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.035e-06 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175820+0000", 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5272999999999998e-05 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175836+0000", 2024-09-15T01:32:36.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6350000000000001e-05 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175844+0000", 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8620000000000003e-06 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175890+0000", 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.617e-05 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:32:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176032+0000", 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141426 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176738+0000", 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00070667099999999997 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176861+0000", 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012252899999999999 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176905+0000", 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3761000000000003e-05 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176965+0000", 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9910000000000001e-05 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176985+0000", 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0047999999999998e-05 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187276+0000", 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010291664000000001 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187338+0000", 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1535e-05 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:11 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.29773904600000001, 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032033109999999997, 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 11 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176990+0000", 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2200000000000002e-07 2024-09-15T01:32:36.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176995+0000", 2024-09-15T01:32:36.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8040000000000001e-06 2024-09-15T01:32:36.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177006+0000", 2024-09-15T01:32:36.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1276e-05 2024-09-15T01:32:36.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177012+0000", 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9370000000000001e-06 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177055+0000", 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2472999999999998e-05 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177197+0000", 2024-09-15T01:32:36.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142519 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177567+0000", 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036945000000000001 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177659+0000", 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1928999999999993e-05 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177694+0000", 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4885000000000002e-05 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177722+0000", 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8334e-05 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177733+0000", 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1491000000000001e-05 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:32:36.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208963+0000", 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031230106000000001 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209022+0000", 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8984000000000003e-05 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:12 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.29698877800000001, 2024-09-15T01:32:36.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033440417, 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 12 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:32:36.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:32:36.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2500000000000005e-07 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177745+0000", 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7269999999999998e-06 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177755+0000", 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9000000000000001e-06 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177760+0000", 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1150000000000002e-06 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177801+0000", 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0802e-05 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177877+0000", 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6604e-05 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209067+0000", 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031189505999999999 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209144+0000", 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6728000000000001e-05 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211180+0000", 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020363099999999999 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=2, reqid=client.4477.0:23, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.186250+0000", 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.28847824999999999, 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001265181, 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186250+0000", 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186250+0000", 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186250+0000", 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8200000000000001e-07 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186267+0000", 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7073000000000001e-05 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186574+0000", 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030691600000000001 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186580+0000", 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6899999999999997e-06 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186878+0000", 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029790600000000002 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186885+0000", 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2540000000000003e-06 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186971+0000", 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5987000000000002e-05 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186999+0000", 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8195e-05 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187016+0000", 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6546e-05 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187465+0000", 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044907900000000001 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187515+0000", 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0253000000000001e-05 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=4, reqid=client.4477.0:25, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.186288+0000", 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.28844065699999999, 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001457444, 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186288+0000", 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186288+0000", 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186288+0000", 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4100000000000001e-07 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186317+0000", 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9264e-05 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186617+0000", 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030016999999999999 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186621+0000", 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8820000000000003e-06 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187008+0000", 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00038662499999999998 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187016+0000", 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9780000000000005e-06 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187116+0000", 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9747000000000006e-05 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187123+0000", 2024-09-15T01:32:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9570000000000002e-06 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187173+0000", 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9543000000000003e-05 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187487+0000", 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031407400000000002 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187745+0000", 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000258063 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=5, reqid=client.4477.0:26, at_version=34'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.186318+0000", 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.28841018499999999, 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015593549999999999, 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186318+0000", 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186318+0000", 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186318+0000", 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1899999999999999e-07 2024-09-15T01:32:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186323+0000", 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7659999999999998e-06 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186630+0000", 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030672300000000001 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186634+0000", 2024-09-15T01:32:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7850000000000002e-06 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187181+0000", 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00054745800000000002 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187188+0000", 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0849999999999999e-06 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187273+0000", 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5004000000000001e-05 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187430+0000", 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000156498 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187462+0000", 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2418e-05 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187526+0000", 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3524999999999999e-05 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187878+0000", 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035197400000000002 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=6, reqid=client.4477.0:40, at_version=34'5, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.186324+0000", 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.28840462100000003, 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001818357, 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186324+0000", 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186324+0000", 2024-09-15T01:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186324+0000", 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-08 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186358+0000", 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4090000000000001e-05 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186642+0000", 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028416800000000001 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186646+0000", 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4539999999999999e-06 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187311+0000", 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00066553499999999998 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187318+0000", 2024-09-15T01:32:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6209999999999998e-06 2024-09-15T01:32:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187444+0000", 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000126468 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187465+0000", 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0809999999999999e-05 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187508+0000", 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3337000000000002e-05 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187540+0000", 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1285999999999998e-05 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.188142+0000", 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00060249499999999998 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=7, reqid=client.4477.0:41, at_version=34'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.186359+0000", 2024-09-15T01:32:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.28836942199999999, 2024-09-15T01:32:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021795830000000001, 2024-09-15T01:32:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186359+0000", 2024-09-15T01:32:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186359+0000", 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186359+0000", 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.06e-07 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.186364+0000", 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187700+0000", 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013361670000000001 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187706+0000", 2024-09-15T01:32:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4829999999999999e-06 2024-09-15T01:32:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187744+0000", 2024-09-15T01:32:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8495999999999999e-05 2024-09-15T01:32:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187751+0000", 2024-09-15T01:32:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8079999999999999e-06 2024-09-15T01:32:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187843+0000", 2024-09-15T01:32:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1954999999999996e-05 2024-09-15T01:32:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187850+0000", 2024-09-15T01:32:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.43e-06 2024-09-15T01:32:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187877+0000", 2024-09-15T01:32:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6857999999999999e-05 2024-09-15T01:32:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187970+0000", 2024-09-15T01:32:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2318999999999995e-05 2024-09-15T01:32:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.188539+0000", 2024-09-15T01:32:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00056895700000000003 2024-09-15T01:32:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpRead(3.0s2 34/31 ECSubRead(tid=9, to_read={3:0dcf4e6a:::smithi17529310-7:head=0,4096,0}, subchunks={3:0dcf4e6a:::smithi17529310-7:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:32:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.188271+0000", 2024-09-15T01:32:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.28645742699999999, 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018723139999999999, 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.188271+0000", 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.188271+0000", 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.188271+0000", 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3300000000000003e-07 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.188278+0000", 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5540000000000003e-06 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189998+0000", 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001719978 2024-09-15T01:32:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190006+0000", 2024-09-15T01:32:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7789999999999996e-06 2024-09-15T01:32:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190054+0000", 2024-09-15T01:32:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8322000000000003e-05 2024-09-15T01:32:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190143+0000", 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9247999999999997e-05 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=12, reqid=client.4477.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.28478456499999999, 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022201101000000001, 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.75e-07 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189957+0000", 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2531000000000001e-05 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211941+0000", 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021983546999999999 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211947+0000", 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7329999999999997e-06 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211977+0000", 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9607e-05 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211982+0000", 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6290000000000001e-06 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212053+0000", 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1890000000000005e-05 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212060+0000", 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3339999999999998e-06 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212087+0000", 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7314e-05 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212098+0000", 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1141999999999999e-05 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212145+0000", 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699000000000001e-05 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 34/31 ECSubWrite(tid=1, reqid=client.4477.0:34, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.190517+0000", 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.28421129000000001, 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0012675309999999999, 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190517+0000", 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190517+0000", 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190517+0000", 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6300000000000001e-07 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190699+0000", 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000181163 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190779+0000", 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0378999999999997e-05 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190790+0000", 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0985000000000001e-05 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190958+0000", 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016856600000000001 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190967+0000", 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4500000000000004e-06 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191097+0000", 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013003900000000001 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191260+0000", 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016339100000000001 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191452+0000", 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019197999999999999 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191480+0000", 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8143000000000001e-05 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191785+0000", 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030417200000000001 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:70 3.5s0 3:af8bec4d:::smithi17529310-24:head [append 0~393216,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.207051+0000", 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.26767698000000001, 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026246640000000001, 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 70 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207051+0000", 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207051+0000", 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207053+0000", 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.409e-06 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207264+0000", 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021139999999999999 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207277+0000", 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2622e-05 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207282+0000", 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2299999999999999e-06 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4928000000000002e-05 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207420+0000", 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103205 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207939+0000", 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051830300000000001 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208010+0000", 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1131999999999996e-05 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208016+0000", 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5039999999999999e-06 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208102+0000", 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5364000000000005e-05 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208861+0000", 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000759061 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209619+0000", 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00075831300000000004 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209676+0000", 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7192999999999999e-05 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:71 3.5s0 3:af8bec4d:::smithi17529310-24:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.207290+0000", 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.26743885299999998, 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028160490000000002, 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207290+0000", 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207290+0000", 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207290+0000", 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4000000000000002e-07 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207295+0000", 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4730000000000002e-06 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207300+0000", 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0150000000000003e-06 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207302+0000", 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4449999999999999e-06 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208064+0000", 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00076162599999999995 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208130+0000", 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6816999999999994e-05 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208505+0000", 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037424000000000002 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208583+0000", 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8626000000000002e-05 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208597+0000", 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3696999999999999e-05 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208628+0000", 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0818999999999997e-05 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208865+0000", 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000237292 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210033+0000", 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001167529 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210106+0000", 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2929999999999995e-05 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:72 3.5s0 3:af8bec4d:::smithi17529310-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.26741183099999999, 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.037268951000000002, 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.34e-07 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207319+0000", 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207326+0000", 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4860000000000002e-06 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207330+0000", 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0690000000000003e-06 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208687+0000", 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001356966 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208760+0000", 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2663000000000005e-05 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210380+0000", 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001620659 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210464+0000", 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3387999999999995e-05 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.244586+0000", 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034121925999999997 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpReadReply(3.5s0 34/31 ECSubReadReply(tid=4, attrs_read=0))", 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.209696+0000", 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.26503271900000003, 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001262411, 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209696+0000", 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209696+0000", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209696+0000", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0200000000000001e-07 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209704+0000", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9610000000000005e-06 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209729+0000", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4193999999999999e-05 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209733+0000", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8269999999999997e-06 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210927+0000", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011938389999999999 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210958+0000", 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0888e-05 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.0s2 34/31 ECSubWrite(tid=15, reqid=client.4477.0:74, at_version=34'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.26230108400000002, 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069361051000000007, 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212428+0000", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3399999999999999e-07 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212444+0000", 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5681000000000001e-05 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281357+0000", 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068913881999999996 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281366+0000", 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9740000000000008e-06 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281415+0000", 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8498000000000002e-05 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281421+0000", 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4590000000000003e-06 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281487+0000", 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5590000000000001e-05 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281500+0000", 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3298e-05 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281575+0000", 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5247000000000006e-05 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281597+0000", 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1872999999999999e-05 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281788+0000", 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000191015 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:103 3.5s0 3:a0a9f8ce:::smithi17529310-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.242159+0000", 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.232569058, 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027786159999999998, 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 103 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.242159+0000", 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.242159+0000", 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.242161+0000", 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6360000000000001e-06 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.242483+0000", 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032196200000000002 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.242499+0000", 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6368e-05 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.242505+0000", 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9970000000000004e-06 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.242543+0000", 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7997999999999998e-05 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.242650+0000", 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107182 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.243231+0000", 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00058043000000000005 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.243304+0000", 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2857999999999999e-05 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.243317+0000", 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3039e-05 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.243459+0000", 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014261300000000001 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.244130+0000", 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00067051199999999995 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.244872+0000", 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00074226499999999998 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.244938+0000", 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5755999999999999e-05 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=20, reqid=client.4477.0:131, at_version=0'0, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.282692+0000", 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.19203682499999999, 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01281827, 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.282692+0000", 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.282692+0000", 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.282692+0000", 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.68e-07 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.282702+0000", 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8230000000000006e-06 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295220+0000", 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012517916 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295229+0000", 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.867e-06 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295274+0000", 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5583e-05 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295283+0000", 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1880000000000002e-06 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295360+0000", 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7578999999999997e-05 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295372+0000", 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.207e-05 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295419+0000", 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7289000000000001e-05 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295443+0000", 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3714999999999999e-05 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295510+0000", 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6672000000000001e-05 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s3 34/31 ECSubWrite(tid=136, reqid=client.4477.0:154, at_version=34'9, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.308533+0000", 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.16619582999999999, 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0091997880000000004, 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308533+0000", 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308533+0000", 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308534+0000", 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3599999999999999e-06 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308560+0000", 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5592000000000002e-05 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308586+0000", 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5998000000000001e-05 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308593+0000", 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9019999999999999e-06 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308649+0000", 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5816000000000002e-05 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308659+0000", 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3579999999999992e-06 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308751+0000", 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2588000000000002e-05 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308762+0000", 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0669000000000001e-05 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308803+0000", 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1439000000000001e-05 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308822+0000", 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9021e-05 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.317732+0000", 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089100450000000001 2024-09-15T01:32:36.584 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.585 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.585 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.585 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s3 34/31 ECSubWrite(tid=137, reqid=client.4477.0:154, at_version=0'0, trim_to=0'0, roll_forward_to=34'9))", 2024-09-15T01:32:36.585 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.310906+0000", 2024-09-15T01:32:36.585 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.163822727, 2024-09-15T01:32:36.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0070108710000000001, 2024-09-15T01:32:36.585 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:32:36.585 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:32:36.585 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:32:36.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.310906+0000", 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.310906+0000", 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.310912+0000", 2024-09-15T01:32:36.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0249999999999999e-06 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.310921+0000", 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4960000000000002e-06 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.310936+0000", 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4582e-05 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.310940+0000", 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4070000000000003e-06 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.310986+0000", 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6282999999999997e-05 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.310995+0000", 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9339999999999995e-06 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.311048+0000", 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2612999999999999e-05 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.311057+0000", 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0969999999999995e-06 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.311925+0000", 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00086781300000000005 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.311954+0000", 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9558999999999999e-05 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.317917+0000", 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0059620619999999997 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:32:36.590 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:32:36.591 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:32:36.592 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:32:36.592 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:36.590+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:32:36.593 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:32:36.593 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:32:36.593 INFO:tasks.rados.rados.0.smithi175.stdout:39: finishing write tid 1 to smithi17529310-39 2024-09-15T01:32:36.594 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:32:36.594 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:32:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:32:36.595 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:32:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:32:36.595 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:32:36.596 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:32:36.596 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:32:36.596 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:32:36.597 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:32:36.597 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:32:36.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:32:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:32:36.631 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:32:36.632 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:32:36.650 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:32:36.651 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:32:36.669 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:32:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:32:36.688 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:32:36.689 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:32:36.693 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:36.691+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:36.707 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:32:36.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:32:36.706+0000 7f01777fe640 0 -- 172.21.15.42:0/3730982110 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7f0178194c90 msgr2=0x7f0178197070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:32:36.714 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:32:36.734 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:32:36.734 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:32:36.749 INFO:tasks.rados.rados.0.smithi175.stdout:49: finishing write tid 3 to smithi17529310-49 2024-09-15T01:32:36.750 INFO:tasks.rados.rados.0.smithi175.stdout:49: finishing write tid 4 to smithi17529310-49 2024-09-15T01:32:36.750 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 49 v 13 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-15T01:32:36.750 INFO:tasks.rados.rados.0.smithi175.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-09-15T01:32:36.770 INFO:tasks.rados.rados.0.smithi175.stdout:42: finishing write tid 3 to smithi17529310-42 2024-09-15T01:32:36.770 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 42 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-15T01:32:36.770 INFO:tasks.rados.rados.0.smithi175.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-09-15T01:32:36.793 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:36.791+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:36.808 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:36.808 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:36.817 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:32:36.846 INFO:tasks.rados.rados.0.smithi175.stdout:50: finishing write tid 2 to smithi17529310-50 2024-09-15T01:32:36.897 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:36.895+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:36.912 INFO:tasks.rados.rados.0.smithi175.stdout:35: finishing write tid 2 to smithi17529310-35 2024-09-15T01:32:36.913 INFO:tasks.rados.rados.0.smithi175.stdout:35: finishing write tid 3 to smithi17529310-35 2024-09-15T01:32:36.913 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 35 v 6 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-15T01:32:36.913 INFO:tasks.rados.rados.0.smithi175.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-09-15T01:32:36.918 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:36.918 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:36.920 INFO:tasks.rados.rados.0.smithi175.stdout:47: finishing write tid 2 to smithi17529310-47 2024-09-15T01:32:36.921 INFO:tasks.rados.rados.0.smithi175.stdout:50: finishing write tid 3 to smithi17529310-50 2024-09-15T01:32:36.922 INFO:tasks.rados.rados.0.smithi175.stdout:50: finishing write tid 4 to smithi17529310-50 2024-09-15T01:32:36.923 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 50 v 5 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-15T01:32:36.923 INFO:tasks.rados.rados.0.smithi175.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-09-15T01:32:36.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:32:36.997 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:36.996+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:37.028 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:37.028 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:37.036 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:32:37.099 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:37.097+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:37.137 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:37.137 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:37.145 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:32:37.179 INFO:teuthology.orchestra.run.smithi042.stderr:marked out osd.9. 2024-09-15T01:32:37.181 INFO:tasks.rados.rados.0.smithi175.stdout:39: finishing write tid 2 to smithi17529310-39 2024-09-15T01:32:37.181 INFO:tasks.rados.rados.0.smithi175.stdout:47: finishing write tid 3 to smithi17529310-47 2024-09-15T01:32:37.182 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 47 v 6 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-15T01:32:37.182 INFO:tasks.rados.rados.0.smithi175.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-09-15T01:32:37.182 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-15T01:32:37.183 INFO:tasks.rados.rados.0.smithi175.stdout:39: finishing write tid 3 to smithi17529310-39 2024-09-15T01:32:37.183 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 39 v 10 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-15T01:32:37.183 INFO:tasks.rados.rados.0.smithi175.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-09-15T01:32:37.186 INFO:tasks.rados.rados.0.smithi175.stdout:15: done (11 left) 2024-09-15T01:32:37.186 INFO:tasks.rados.rados.0.smithi175.stdout:35: done (10 left) 2024-09-15T01:32:37.186 INFO:tasks.rados.rados.0.smithi175.stdout:39: done (9 left) 2024-09-15T01:32:37.186 INFO:tasks.rados.rados.0.smithi175.stdout:42: done (8 left) 2024-09-15T01:32:37.186 INFO:tasks.rados.rados.0.smithi175.stdout:43: done (7 left) 2024-09-15T01:32:37.186 INFO:tasks.rados.rados.0.smithi175.stdout:47: done (6 left) 2024-09-15T01:32:37.186 INFO:tasks.rados.rados.0.smithi175.stdout:48: done (5 left) 2024-09-15T01:32:37.186 INFO:tasks.rados.rados.0.smithi175.stdout:49: done (4 left) 2024-09-15T01:32:37.186 INFO:tasks.rados.rados.0.smithi175.stdout:50: done (3 left) 2024-09-15T01:32:37.187 INFO:tasks.rados.rados.0.smithi175.stdout:52: done (2 left) 2024-09-15T01:32:37.187 INFO:tasks.rados.rados.0.smithi175.stdout:53: done (1 left) 2024-09-15T01:32:37.187 INFO:tasks.rados.rados.0.smithi175.stdout:54: read oid 11 snap 0 2024-09-15T01:32:37.187 INFO:tasks.rados.rados.0.smithi175.stdout:54: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T01:32:37.187 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 41 current snap is 1 2024-09-15T01:32:37.187 INFO:tasks.rados.rados.0.smithi175.stdout:55: seq_num 51 ranges {688128=491520,1179648=376832} 2024-09-15T01:32:37.192 INFO:tasks.rados.rados.0.smithi175.stdout:55: writing smithi17529310-41 from 688128 to 1179648 tid 1 2024-09-15T01:32:37.195 INFO:tasks.rados.rados.0.smithi175.stdout:55: writing smithi17529310-41 from 1179648 to 1556480 tid 2 2024-09-15T01:32:37.195 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 48 current snap is 1 2024-09-15T01:32:37.195 INFO:tasks.rados.rados.0.smithi175.stdout:56: seq_num 52 ranges {802816=507904,1310720=376832} 2024-09-15T01:32:37.199 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:37.196+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:37.201 INFO:tasks.rados.rados.0.smithi175.stdout:56: writing smithi17529310-48 from 802816 to 1310720 tid 1 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:56: writing smithi17529310-48 from 1310720 to 1687552 tid 2 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:57: read oid 35 snap -1 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:57: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:58: read oid 45 snap -1 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:58: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:59: read oid 19 snap -1 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:59: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:60: snap_create 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:55: finishing write tid 1 to smithi17529310-41 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:55: finishing write tid 2 to smithi17529310-41 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:55: finishing write tid 3 to smithi17529310-41 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:55: finishing write tid 4 to smithi17529310-41 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 41 v 12 (ObjNum 51 snap 1 seq_num 51) dirty exists 2024-09-15T01:32:37.205 INFO:tasks.rados.rados.0.smithi175.stdout:55: left oid 41 (ObjNum 51 snap 1 seq_num 51) 2024-09-15T01:32:37.206 INFO:tasks.rados.rados.0.smithi175.stdout:56: finishing write tid 1 to smithi17529310-48 2024-09-15T01:32:37.207 INFO:tasks.rados.rados.0.smithi175.stdout:56: finishing write tid 2 to smithi17529310-48 2024-09-15T01:32:37.207 INFO:tasks.rados.rados.0.smithi175.stdout:56: finishing write tid 3 to smithi17529310-48 2024-09-15T01:32:37.208 INFO:tasks.rados.rados.0.smithi175.stdout:57: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-15T01:32:37.223 INFO:tasks.rados.rados.0.smithi175.stdout:56: finishing write tid 4 to smithi17529310-48 2024-09-15T01:32:37.223 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 48 v 13 (ObjNum 52 snap 1 seq_num 52) dirty exists 2024-09-15T01:32:37.223 INFO:tasks.rados.rados.0.smithi175.stdout:56: left oid 48 (ObjNum 52 snap 1 seq_num 52) 2024-09-15T01:32:37.223 INFO:tasks.rados.rados.0.smithi175.stdout:58: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:37.267 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:32:37.300 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:37.299+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:37.368 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:37.368 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:37.377 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:32:37.420 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:37.419+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:37.477 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:37.477 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:37.485 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:32:37.502 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:37.500+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:37.602 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:37.601+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:37.626 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:37.627 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:37.636 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-15T01:32:37.703 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:37.702+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:37.777 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:37.777 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:37.785 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-15T01:32:37.805 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:37.803+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:37.885 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:37.885 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:37.893 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-15T01:32:37.906 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:37.904+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:37.994 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:37.994 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:38.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:32:38.046 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:38.045+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:38.106 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:38.105+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:38.111 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:38.111 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:38.120 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:32:38.181 INFO:tasks.rados.rados.0.smithi175.stdout:55: done (7 left) 2024-09-15T01:32:38.181 INFO:tasks.rados.rados.0.smithi175.stdout:56: done (6 left) 2024-09-15T01:32:38.181 INFO:tasks.rados.rados.0.smithi175.stdout:57: done (5 left) 2024-09-15T01:32:38.181 INFO:tasks.rados.rados.0.smithi175.stdout:58: done (4 left) 2024-09-15T01:32:38.181 INFO:tasks.rados.rados.0.smithi175.stdout:60: done (3 left) 2024-09-15T01:32:38.181 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 13 current snap is 2 2024-09-15T01:32:38.181 INFO:tasks.rados.rados.0.smithi175.stdout:61: seq_num 53 ranges {212992=524288,737280=376832} 2024-09-15T01:32:38.185 INFO:tasks.rados.rados.0.smithi175.stdout:61: writing smithi17529310-13 from 212992 to 737280 tid 1 2024-09-15T01:32:38.187 INFO:tasks.rados.rados.0.smithi175.stdout:61: writing smithi17529310-13 from 737280 to 1114112 tid 2 2024-09-15T01:32:38.188 INFO:tasks.rados.rados.0.smithi175.stdout:62: setattr oid 32 current snap is 2 2024-09-15T01:32:38.188 INFO:tasks.rados.rados.0.smithi175.stdout:63: delete oid 15 current snap is 2 2024-09-15T01:32:38.192 INFO:tasks.rados.rados.0.smithi175.stdout:61: finishing write tid 1 to smithi17529310-13 2024-09-15T01:32:38.192 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 32 v 8 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-15T01:32:38.192 INFO:tasks.rados.rados.0.smithi175.stdout:61: finishing write tid 2 to smithi17529310-13 2024-09-15T01:32:38.192 INFO:tasks.rados.rados.0.smithi175.stdout:61: finishing write tid 3 to smithi17529310-13 2024-09-15T01:32:38.193 INFO:tasks.rados.rados.0.smithi175.stdout:61: finishing write tid 4 to smithi17529310-13 2024-09-15T01:32:38.194 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 13 v 17 (ObjNum 53 snap 2 seq_num 53) dirty exists 2024-09-15T01:32:38.194 INFO:tasks.rados.rados.0.smithi175.stdout:61: left oid 13 (ObjNum 53 snap 2 seq_num 53) 2024-09-15T01:32:38.207 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:38.206+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:38.263 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:38.263 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:38.271 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:32:38.315 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:38.314+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:38.367 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:38.368 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:38.376 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:32:38.409 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:38.407+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:38.510 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:38.508+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:38.513 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:38.513 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:38.521 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:32:38.611 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:38.609+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:38.617 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:38.617 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:38.625 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:32:38.711 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:38.709+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:38.721 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:38.721 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:38.729 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:32:38.745 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:32:38.813 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:38.810+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:38.825 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:38.825 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:38.834 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:32:38.885 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:32:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:32:38.905 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:32:38.905 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:32:38.913 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:38.912+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:38.925 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:32:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:32:38.931 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:38.931 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:38.939 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:32:38.945 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:32:38.945 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:32:38.963 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:32:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:32:38.983 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:32:38.983 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:32:39.003 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:32:39.003 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:32:39.014 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:39.013+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:39.023 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:32:39.023 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:32:39.041 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:32:39.041 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:32:39.060 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:32:39.060 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 33/31 ECSubWrite(tid=2, reqid=client.4477.0:1, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.176343+0000", 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8992497899999998, 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096529600000000004, 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176343+0000", 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176343+0000", 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176343+0000", 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3999999999999998e-07 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176405+0000", 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1896000000000005e-05 2024-09-15T01:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176489+0000", 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3608000000000003e-05 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176500+0000", 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0685e-05 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176570+0000", 2024-09-15T01:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0057999999999999e-05 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176580+0000", 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0309000000000001e-05 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176704+0000", 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012443600000000001 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176750+0000", 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5213999999999998e-05 2024-09-15T01:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176809+0000", 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9477e-05 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176988+0000", 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178636 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177309+0000", 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032073699999999998 2024-09-15T01:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 33/31 ECSubWrite(tid=3, reqid=client.4477.0:2, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.176414+0000", 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.899179245, 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010880270000000001, 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176414+0000", 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176414+0000", 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176414+0000", 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7100000000000002e-07 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176430+0000", 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5682e-05 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176522+0000", 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2359000000000001e-05 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176527+0000", 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176766+0000", 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000238411 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176802+0000", 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5769000000000003e-05 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176941+0000", 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013990500000000001 2024-09-15T01:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176969+0000", 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7251999999999999e-05 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177029+0000", 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9933999999999999e-05 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177048+0000", 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9123e-05 2024-09-15T01:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177502+0000", 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045398500000000001 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 34/31 ECSubWrite(tid=0, reqid=client.4477.0:13, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179137+0000", 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8964556809999999, 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017856180000000001, 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179137+0000", 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179137+0000", 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179138+0000", 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3900000000000004e-07 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179179+0000", 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0599999999999998e-05 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179867+0000", 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068798899999999996 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179876+0000", 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180320+0000", 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044354399999999998 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180328+0000", 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7649999999999993e-06 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180424+0000", 2024-09-15T01:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6311000000000007e-05 2024-09-15T01:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180437+0000", 2024-09-15T01:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3001e-05 2024-09-15T01:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180528+0000", 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1378000000000005e-05 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180623+0000", 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4455000000000003e-05 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180923+0000", 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030002300000000001 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=1, reqid=client.4477.0:28, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8964104690000001, 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018499090000000001, 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179245+0000", 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2150000000000006e-05 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180346+0000", 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001101196 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180357+0000", 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175000000000001e-05 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180408+0000", 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0590999999999997e-05 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180417+0000", 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8060000000000001e-06 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180553+0000", 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136324 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180684+0000", 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000130853 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180745+0000", 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0971000000000003e-05 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180919+0000", 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017386800000000001 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181032+0000", 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113578 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 34/31 ECSubWrite(tid=2, reqid=client.4477.0:14, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.896344134, 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018048179999999999, 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179259+0000", 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0104e-05 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180590+0000", 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013307779999999999 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180598+0000", 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180692+0000", 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3549999999999997e-05 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180700+0000", 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6330000000000002e-06 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180793+0000", 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967000000000006e-05 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180801+0000", 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4580000000000004e-06 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180836+0000", 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5732999999999998e-05 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180858+0000", 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.126e-05 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181054+0000", 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019604099999999999 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 33/31 ECSubWrite(tid=4, reqid=client.4477.0:2, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179837+0000", 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8957558089999997, 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00111943, 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179837+0000", 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179837+0000", 2024-09-15T01:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179838+0000", 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.096e-06 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179853+0000", 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4919999999999999e-05 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179893+0000", 2024-09-15T01:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9502999999999998e-05 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179902+0000", 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179998+0000", 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6417999999999997e-05 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180008+0000", 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8980000000000008e-06 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180107+0000", 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9267999999999999e-05 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180121+0000", 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3715e-05 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180201+0000", 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0383999999999995e-05 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180934+0000", 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00073209700000000002 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180957+0000", 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3104000000000001e-05 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 34/31 ECSubWrite(tid=4, reqid=client.4477.0:31, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.180672+0000", 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.894920725, 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014757450000000001, 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180672+0000", 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180672+0000", 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180673+0000", 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2799999999999996e-07 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181385+0000", 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071188100000000004 2024-09-15T01:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181419+0000", 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4078999999999997e-05 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181425+0000", 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5989999999999998e-06 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181488+0000", 2024-09-15T01:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2407000000000001e-05 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181498+0000", 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003999999999999e-05 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181638+0000", 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014010499999999999 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181679+0000", 2024-09-15T01:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0672e-05 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181759+0000", 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0067000000000004e-05 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181779+0000", 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0336e-05 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182148+0000", 2024-09-15T01:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000369067 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=0, reqid=client.4477.0:16, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.181909+0000", 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8936843190000001, 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00076923200000000001, 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181909+0000", 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181909+0000", 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181909+0000", 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2500000000000001e-07 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181978+0000", 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8928000000000001e-05 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182070+0000", 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2204e-05 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182077+0000", 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0210000000000002e-06 2024-09-15T01:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182122+0000", 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4582000000000001e-05 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182131+0000", 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8510000000000005e-06 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182261+0000", 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000129875 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182294+0000", 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3291999999999999e-05 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182356+0000", 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1696e-05 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182487+0000", 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013164000000000001 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182678+0000", 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019071800000000001 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=1, reqid=client.4477.0:17, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.183115+0000", 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.892477929, 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00064692799999999996, 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183115+0000", 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183115+0000", 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183116+0000", 2024-09-15T01:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000003e-07 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183134+0000", 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8031000000000001e-05 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183179+0000", 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5466999999999997e-05 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183187+0000", 2024-09-15T01:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1899999999999995e-06 2024-09-15T01:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183249+0000", 2024-09-15T01:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1975000000000002e-05 2024-09-15T01:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183276+0000", 2024-09-15T01:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7056e-05 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183365+0000", 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8863000000000006e-05 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183397+0000", 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2082999999999998e-05 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183439+0000", 2024-09-15T01:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1269000000000001e-05 2024-09-15T01:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183458+0000", 2024-09-15T01:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9570000000000001e-05 2024-09-15T01:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183762+0000", 2024-09-15T01:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030372599999999999 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 34/31 ECSubWrite(tid=6, reqid=client.4477.0:43, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.187004+0000", 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.888589149, 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011234019999999999, 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187004+0000", 2024-09-15T01:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187004+0000", 2024-09-15T01:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187005+0000", 2024-09-15T01:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0470000000000001e-06 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187114+0000", 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010932700000000001 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187177+0000", 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2597999999999999e-05 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187185+0000", 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1780000000000003e-06 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187279+0000", 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3690999999999995e-05 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187288+0000", 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2799999999999992e-06 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187629+0000", 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034080200000000002 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187682+0000", 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2848e-05 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187752+0000", 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9906000000000005e-05 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188095+0000", 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000343297 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188127+0000", 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2428000000000002e-05 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=2, reqid=client.4477.0:17, at_version=0'0, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.187733+0000", 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.887859626, 2024-09-15T01:32:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010603889999999999, 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187733+0000", 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187733+0000", 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188139+0000", 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040583200000000001 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188168+0000", 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8878999999999999e-05 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188207+0000", 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9385999999999999e-05 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188219+0000", 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1521e-05 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188274+0000", 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5324000000000001e-05 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188284+0000", 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8239999999999995e-06 2024-09-15T01:32:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188391+0000", 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106909 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188449+0000", 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8193999999999999e-05 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188505+0000", 2024-09-15T01:32:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6033999999999999e-05 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188615+0000", 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109959 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188794+0000", 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017852699999999999 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 34/31 ECSubWrite(tid=7, reqid=client.4477.0:44, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.188541+0000", 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8870524949999998, 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00064089400000000003, 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188541+0000", 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188541+0000", 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188541+0000", 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7000000000000004e-07 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188561+0000", 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9981999999999999e-05 2024-09-15T01:32:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188603+0000", 2024-09-15T01:32:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1560000000000002e-05 2024-09-15T01:32:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188615+0000", 2024-09-15T01:32:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1708999999999999e-05 2024-09-15T01:32:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188683+0000", 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8598999999999998e-05 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188692+0000", 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6279999999999994e-06 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188855+0000", 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000163258 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188882+0000", 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6937e-05 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188925+0000", 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2873000000000001e-05 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.189153+0000", 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000228125 2024-09-15T01:32:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.189181+0000", 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8453e-05 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=8, reqid=client.4477.0:55, at_version=34'3, trim_to=0'0, roll_forward_to=34'1))", 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.195377+0000", 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8802161389999998, 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00063860499999999999, 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.195377+0000", 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.195377+0000", 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.195378+0000", 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4839999999999999e-06 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.195467+0000", 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8471e-05 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.195511+0000", 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4521000000000003e-05 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.195519+0000", 2024-09-15T01:32:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8110000000000003e-06 2024-09-15T01:32:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.195563+0000", 2024-09-15T01:32:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4119000000000002e-05 2024-09-15T01:32:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.195570+0000", 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7959999999999998e-06 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.195695+0000", 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124828 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.195704+0000", 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3400000000000004e-06 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.195773+0000", 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9040000000000003e-05 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.195794+0000", 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0509e-05 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.196015+0000", 2024-09-15T01:32:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022168599999999999 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=16, reqid=client.4477.0:76, at_version=34'5, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.214947+0000", 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8606460839999999, 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071814299999999997, 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.214947+0000", 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.214947+0000", 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.214948+0000", 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4810000000000001e-06 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215065+0000", 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116551 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215104+0000", 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9462000000000003e-05 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215113+0000", 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2689999999999999e-06 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215157+0000", 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4127e-05 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215163+0000", 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4670000000000001e-06 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215295+0000", 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013140999999999999 2024-09-15T01:32:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215304+0000", 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0399999999999998e-06 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215379+0000", 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5007999999999997e-05 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215395+0000", 2024-09-15T01:32:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5897999999999999e-05 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215665+0000", 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027043 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=4, reqid=client.4477.0:88, at_version=34'3, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.226933+0000", 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8486601999999999, 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010392190000000001, 2024-09-15T01:32:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.226933+0000", 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.226933+0000", 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.226934+0000", 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.077e-06 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227240+0000", 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030573500000000001 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227270+0000", 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.021e-05 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227277+0000", 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1629999999999999e-06 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227326+0000", 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9227000000000001e-05 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227335+0000", 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9609999999999994e-06 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227508+0000", 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017294000000000001 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227520+0000", 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1963e-05 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227623+0000", 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102663 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227724+0000", 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010162600000000001 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227972+0000", 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024765399999999998 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=5, reqid=client.4477.0:89, at_version=34'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.227505+0000", 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8480883380000002, 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00064992299999999995, 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227505+0000", 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227505+0000", 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227505+0000", 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1800000000000005e-07 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227523+0000", 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7708999999999999e-05 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227558+0000", 2024-09-15T01:32:39.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4310000000000002e-05 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227564+0000", 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4690000000000002e-06 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227610+0000", 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6440000000000003e-05 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227620+0000", 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4029999999999996e-06 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227705+0000", 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5518000000000004e-05 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227713+0000", 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.627e-06 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227739+0000", 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6342999999999998e-05 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227755+0000", 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6164e-05 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.228155+0000", 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039912200000000002 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=18, reqid=client.4477.0:91, at_version=34'7, trim_to=0'0, roll_forward_to=34'5))", 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.230472+0000", 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8451214089999999, 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00078053499999999995, 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230472+0000", 2024-09-15T01:32:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230472+0000", 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230472+0000", 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7800000000000001e-07 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230677+0000", 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000204157 2024-09-15T01:32:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230711+0000", 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4150999999999999e-05 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230718+0000", 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4869999999999996e-06 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230760+0000", 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1587e-05 2024-09-15T01:32:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230768+0000", 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1640000000000001e-06 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230905+0000", 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000137269 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230916+0000", 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1104000000000001e-05 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.231004+0000", 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7379999999999993e-05 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.231016+0000", 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1934e-05 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.231252+0000", 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236524 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 34/31 ECSubWrite(tid=6, reqid=client.4477.0:118, at_version=34'3, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.263014+0000", 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8125787610000001, 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001181788, 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263014+0000", 2024-09-15T01:32:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263014+0000", 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263016+0000", 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.424e-06 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263549+0000", 2024-09-15T01:32:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00053332100000000001 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263582+0000", 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2926000000000003e-05 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263591+0000", 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263633+0000", 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2271999999999998e-05 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263640+0000", 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8190000000000003e-06 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263822+0000", 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018233 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263831+0000", 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263930+0000", 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8944999999999995e-05 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263964+0000", 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4173e-05 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.264196+0000", 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023164099999999999 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 34/31 ECSubWrite(tid=7, reqid=client.4477.0:119, at_version=34'4, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.263660+0000", 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8119331719999998, 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065152199999999997, 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263660+0000", 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263660+0000", 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263660+0000", 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0400000000000002e-07 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263674+0000", 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3579e-05 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263697+0000", 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3499999999999999e-05 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263703+0000", 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-09-15T01:32:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263869+0000", 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016557099999999999 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263878+0000", 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263950+0000", 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496999999999994e-05 2024-09-15T01:32:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263956+0000", 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.305e-06 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263980+0000", 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4036000000000002e-05 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263991+0000", 2024-09-15T01:32:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1770000000000001e-05 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.264311+0000", 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031986800000000001 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 34/31 ECSubWrite(tid=23, reqid=client.4477.0:150, at_version=34'3, trim_to=0'0, roll_forward_to=34'1))", 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.307830+0000", 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.7677630930000001, 2024-09-15T01:32:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070952099999999996, 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307830+0000", 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307830+0000", 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307832+0000", 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2900000000000001e-06 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307956+0000", 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123354 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307991+0000", 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5608000000000002e-05 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307999+0000", 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0790000000000001e-06 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308047+0000", 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7756000000000002e-05 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308054+0000", 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5050000000000004e-06 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308144+0000", 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0972999999999998e-05 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308152+0000", 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308239+0000", 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6976000000000002e-05 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308258+0000", 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8961e-05 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308539+0000", 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028192700000000001 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.185 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:39.186 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:32:39.187 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:32:39.187 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:32:39.187 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:32:39.188 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-15T01:32:39.188 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:32:39.188 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:32:39.189 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:32:39.189 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:32:39.189 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:32:39.190 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:32:39.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:32:39.158+0000 7fe9a3fff640 0 -- 172.21.15.42:0/4073907207 >> [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] conn(0x7fe9a4198110 msgr2=0x7fe9a419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-15T01:32:39.190 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:39.185+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:39.191 INFO:tasks.rados.rados.0.smithi175.stdout:30: finishing write tid 3 to smithi17529310-30 2024-09-15T01:32:39.191 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 30 v 4 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-15T01:32:39.191 INFO:tasks.rados.rados.0.smithi175.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-09-15T01:32:39.193 INFO:tasks.rados.rados.0.smithi175.stdout:30: done (5 left) 2024-09-15T01:32:39.193 INFO:tasks.rados.rados.0.smithi175.stdout:61: done (4 left) 2024-09-15T01:32:39.193 INFO:tasks.rados.rados.0.smithi175.stdout:62: done (3 left) 2024-09-15T01:32:39.193 INFO:tasks.rados.rados.0.smithi175.stdout:63: done (2 left) 2024-09-15T01:32:39.193 INFO:tasks.rados.rados.0.smithi175.stdout:64: rollback oid 21 current snap is 2 2024-09-15T01:32:39.193 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 21 to 1 2024-09-15T01:32:39.193 INFO:tasks.rados.rados.0.smithi175.stdout:65: read oid 22 snap 1 2024-09-15T01:32:39.193 INFO:tasks.rados.rados.0.smithi175.stdout:65: expect deleted 2024-09-15T01:32:39.193 INFO:tasks.rados.rados.0.smithi175.stdout:66: copy_from oid 1 from oid 35 current snap is 2 2024-09-15T01:32:39.194 INFO:tasks.rados.rados.0.smithi175.stdout:67: snap_create 2024-09-15T01:32:39.196 INFO:tasks.rados.rados.0.smithi175.stdout:54: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T01:32:39.202 INFO:tasks.rados.rados.0.smithi175.stdout:64: finishing rollback tid 0 to smithi17529310-21 2024-09-15T01:32:39.202 INFO:tasks.rados.rados.0.smithi175.stdout:64: finishing rollback tid 1 to smithi17529310-21 2024-09-15T01:32:39.202 INFO:tasks.rados.rados.0.smithi175.stdout:59: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-15T01:32:39.202 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:32:39.211 INFO:tasks.rados.rados.0.smithi175.stdout:64: finishing rollback tid 2 to smithi17529310-21 2024-09-15T01:32:39.211 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 21 v 14 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-15T01:32:39.211 INFO:tasks.rados.rados.0.smithi175.stdout:66: finishing copy_from racing read to smithi17529310-1 2024-09-15T01:32:39.211 INFO:tasks.rados.rados.0.smithi175.stdout:66: finishing copy_from to smithi17529310-1 2024-09-15T01:32:39.212 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 1 v 6 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-15T01:32:39.288 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:39.286+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:39.288 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:32:39.289 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:39.296 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-15T01:32:39.380 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:32:39.380 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:32:39.389 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:39.387+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:39.393 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:39.393 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:39.393 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:32:39.393 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:32:39.393 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:39.401 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-15T01:32:39.489 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:39.488+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:39.540 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:32:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:32:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:32:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:32:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.057763345000001, 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27210968699999999, 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479166+0000", 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4729999999999999e-06 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479172+0000", 2024-09-15T01:32:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7460000000000003e-06 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479186+0000", 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4011e-05 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479211+0000", 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4658e-05 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479249+0000", 2024-09-15T01:32:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7706000000000001e-05 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479255+0000", 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9669999999999998e-06 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750042+0000", 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27078760899999998 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750204+0000", 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016171800000000001 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750337+0000", 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000132674 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750470+0000", 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013314699999999999 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750485+0000", 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5288e-05 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750515+0000", 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0179e-05 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750582+0000", 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6890999999999994e-05 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751212+0000", 2024-09-15T01:32:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062971900000000005 2024-09-15T01:32:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751249+0000", 2024-09-15T01:32:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7072e-05 2024-09-15T01:32:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751274+0000", 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4828999999999999e-05 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.146116+0000", 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.390811486, 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049771340000000002, 2024-09-15T01:32:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52 2024-09-15T01:32:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146116+0000", 2024-09-15T01:32:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146116+0000", 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146116+0000", 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5199999999999998e-07 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146120+0000", 2024-09-15T01:32:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4350000000000001e-06 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146126+0000", 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6579999999999996e-06 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.146130+0000", 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9709999999999998e-06 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150406+0000", 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042754960000000002 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150503+0000", 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7405999999999995e-05 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150614+0000", 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110476 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150714+0000", 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010037799999999999 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150724+0000", 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3810000000000005e-06 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150756+0000", 2024-09-15T01:32:39.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2889999999999999e-05 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.150931+0000", 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017436700000000001 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.151051+0000", 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012015600000000001 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.151074+0000", 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3402999999999999e-05 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.151093+0000", 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8765000000000001e-05 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269156+0000", 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.267771805999999, 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010045861, 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 72 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269156+0000", 2024-09-15T01:32:39.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269156+0000", 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269156+0000", 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1199999999999999e-07 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269218+0000", 2024-09-15T01:32:39.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.19e-05 2024-09-15T01:32:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269222+0000", 2024-09-15T01:32:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.673e-06 2024-09-15T01:32:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269225+0000", 2024-09-15T01:32:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.441e-06 2024-09-15T01:32:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271274+0000", 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020488540000000001 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271323+0000", 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9258999999999997e-05 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271394+0000", 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0745000000000001e-05 2024-09-15T01:32:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271483+0000", 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9605999999999996e-05 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271491+0000", 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5800000000000003e-06 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271554+0000", 2024-09-15T01:32:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3157000000000005e-05 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271765+0000", 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021087699999999999 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279154+0000", 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073888729999999998 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279181+0000", 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7370000000000001e-05 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279202+0000", 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0313999999999999e-05 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.564 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269228+0000", 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.267699458999999, 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010072035, 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 73 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269228+0000", 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269228+0000", 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269228+0000", 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3099999999999999e-07 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269230+0000", 2024-09-15T01:32:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.137e-06 2024-09-15T01:32:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269234+0000", 2024-09-15T01:32:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0599999999999999e-06 2024-09-15T01:32:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269235+0000", 2024-09-15T01:32:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.922e-06 2024-09-15T01:32:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271530+0000", 2024-09-15T01:32:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0022941419999999999 2024-09-15T01:32:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271573+0000", 2024-09-15T01:32:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3789999999999999e-05 2024-09-15T01:32:39.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271653+0000", 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9827000000000007e-05 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271737+0000", 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3950999999999996e-05 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271742+0000", 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2449999999999998e-06 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271768+0000", 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5581000000000001e-05 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271922+0000", 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153871 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279240+0000", 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073177279999999999 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279275+0000", 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5179999999999999e-05 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279300+0000", 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.527e-05 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269238+0000", 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.267689627999999, 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010290084999999999, 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 74 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.572 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269238+0000", 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269238+0000", 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269238+0000", 2024-09-15T01:32:39.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269240+0000", 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4310000000000001e-06 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269243+0000", 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269245+0000", 2024-09-15T01:32:39.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8679999999999999e-06 2024-09-15T01:32:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272014+0000", 2024-09-15T01:32:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027686450000000001 2024-09-15T01:32:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272069+0000", 2024-09-15T01:32:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5031000000000002e-05 2024-09-15T01:32:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272175+0000", 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106455 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272298+0000", 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122891 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272304+0000", 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3510000000000001e-06 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272329+0000", 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4905999999999998e-05 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272892+0000", 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00056393100000000005 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279488+0000", 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065956269999999997 2024-09-15T01:32:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279511+0000", 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2527e-05 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279528+0000", 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7294000000000001e-05 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269248+0000", 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.267679838999999, 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010540288, 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 75 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269248+0000", 2024-09-15T01:32:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269248+0000", 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269248+0000", 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.04e-07 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269301+0000", 2024-09-15T01:32:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2757000000000003e-05 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269305+0000", 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3780000000000003e-06 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269309+0000", 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.738e-06 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272346+0000", 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030372089999999999 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272414+0000", 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8232000000000007e-05 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272543+0000", 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012883799999999999 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272646+0000", 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102948 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272651+0000", 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6879999999999998e-06 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272703+0000", 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1901000000000003e-05 2024-09-15T01:32:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272897+0000", 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000194472 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279745+0000", 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068477929999999996 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279770+0000", 2024-09-15T01:32:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4820999999999998e-05 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279788+0000", 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8309e-05 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269311+0000", 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.267616006000001, 2024-09-15T01:32:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010539287, 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 76 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269311+0000", 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269311+0000", 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269312+0000", 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9799999999999999e-07 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269315+0000", 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7889999999999999e-06 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269318+0000", 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0070000000000002e-06 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269320+0000", 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0540000000000002e-06 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272676+0000", 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033561340000000002 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272730+0000", 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3983999999999997e-05 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272797+0000", 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7156999999999994e-05 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272866+0000", 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9453000000000001e-05 2024-09-15T01:32:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272872+0000", 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9660000000000001e-06 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272895+0000", 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2764000000000001e-05 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272915+0000", 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0361e-05 2024-09-15T01:32:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279812+0000", 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068969000000000001 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279833+0000", 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1055000000000001e-05 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279851+0000", 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7365000000000001e-05 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:78 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.280120+0000", 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.256807813999998, 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082697889999999996, 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280120+0000", 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280120+0000", 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280120+0000", 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280162+0000", 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1817999999999998e-05 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280167+0000", 2024-09-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280170+0000", 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5310000000000002e-06 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280390+0000", 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021957699999999999 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280462+0000", 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2725000000000006e-05 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280551+0000", 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8246999999999997e-05 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280646+0000", 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5063999999999996e-05 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280651+0000", 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280706+0000", 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4815000000000003e-05 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281099+0000", 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000392982 2024-09-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288289+0000", 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071901609999999996 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288336+0000", 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7593999999999999e-05 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288389+0000", 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3022999999999997e-05 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.280175+0000", 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.256752405, 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082971350000000006, 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 79 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280175+0000", 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280175+0000", 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280175+0000", 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280217+0000", 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1483000000000002e-05 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280222+0000", 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6260000000000003e-06 2024-09-15T01:32:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280224+0000", 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5950000000000001e-06 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280700+0000", 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047627200000000002 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280804+0000", 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103742 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280917+0000", 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112908 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281046+0000", 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000128555 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281059+0000", 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.362e-05 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281110+0000", 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0457000000000002e-05 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281671+0000", 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00056108199999999997 2024-09-15T01:32:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288422+0000", 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00675145 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288451+0000", 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9133e-05 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288472+0000", 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0761999999999999e-05 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.280228+0000", 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.256699867999998, 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085368890000000006, 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 80 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280228+0000", 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280228+0000", 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280228+0000", 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8800000000000003e-07 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280281+0000", 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2437e-05 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280286+0000", 2024-09-15T01:32:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.835e-06 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280289+0000", 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0790000000000001e-06 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281125+0000", 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00083510599999999996 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281204+0000", 2024-09-15T01:32:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9025999999999999e-05 2024-09-15T01:32:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281350+0000", 2024-09-15T01:32:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000146866 2024-09-15T01:32:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281512+0000", 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000161359 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281518+0000", 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1229999999999997e-06 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281573+0000", 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.541e-05 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281675+0000", 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101724 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288695+0000", 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070196870000000001 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288734+0000", 2024-09-15T01:32:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9193000000000002e-05 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288764+0000", 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0555999999999997e-05 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:99 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.395716+0000", 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.141211726000002, 2024-09-15T01:32:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052892579999999998, 2024-09-15T01:32:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 99 2024-09-15T01:32:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395716+0000", 2024-09-15T01:32:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395716+0000", 2024-09-15T01:32:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395716+0000", 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5699999999999999e-07 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395747+0000", 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1254999999999998e-05 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395751+0000", 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9430000000000002e-06 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395753+0000", 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2960000000000001e-06 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400073+0000", 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004319097 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400142+0000", 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9507999999999999e-05 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400266+0000", 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124287 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400411+0000", 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014435199999999999 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400416+0000", 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5260000000000001e-06 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400463+0000", 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6456999999999999e-05 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400586+0000", 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012369400000000001 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400961+0000", 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037495999999999997 2024-09-15T01:32:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400983+0000", 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1437000000000002e-05 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401005+0000", 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2189e-05 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:100 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.395756+0000", 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.141171165999999, 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0055617000000000002, 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 100 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395756+0000", 2024-09-15T01:32:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395756+0000", 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395757+0000", 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395759+0000", 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2309999999999999e-06 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395762+0000", 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9500000000000001e-06 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395764+0000", 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9580000000000001e-06 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400609+0000", 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048449870000000003 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400678+0000", 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9764999999999994e-05 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400806+0000", 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012723899999999999 2024-09-15T01:32:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400928+0000", 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012266800000000001 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400935+0000", 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4570000000000002e-06 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400962+0000", 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7223e-05 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401017+0000", 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4868999999999999e-05 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401246+0000", 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022941300000000001 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401287+0000", 2024-09-15T01:32:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0193e-05 2024-09-15T01:32:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401318+0000", 2024-09-15T01:32:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1452000000000003e-05 2024-09-15T01:32:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:145 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~28] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:19.474868+0000", 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.062059374, 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071687239999999996, 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474868+0000", 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474868+0000", 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474869+0000", 2024-09-15T01:32:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3399999999999999e-07 2024-09-15T01:32:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474873+0000", 2024-09-15T01:32:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-09-15T01:32:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474883+0000", 2024-09-15T01:32:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0115000000000001e-05 2024-09-15T01:32:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474887+0000", 2024-09-15T01:32:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7000000000000002e-06 2024-09-15T01:32:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474910+0000", 2024-09-15T01:32:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099e-05 2024-09-15T01:32:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474961+0000", 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1285000000000001e-05 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475027+0000", 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5427999999999998e-05 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475105+0000", 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8459000000000002e-05 2024-09-15T01:32:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475133+0000", 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8371999999999999e-05 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475163+0000", 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9181999999999999e-05 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475179+0000", 2024-09-15T01:32:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6069000000000001e-05 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.481979+0000", 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00680038 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.482007+0000", 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7648e-05 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.482037+0000", 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0048000000000001e-05 2024-09-15T01:32:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:7 3.3s0 3:c07cd22b:::smithi17529310-3:head [append 0~49152,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.175606+0000", 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.361320992, 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011071667, 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175606+0000", 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175606+0000", 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175610+0000", 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6239999999999999e-06 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175737+0000", 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012691699999999999 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175767+0000", 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0029999999999999e-05 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175781+0000", 2024-09-15T01:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4173e-05 2024-09-15T01:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175903+0000", 2024-09-15T01:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122031 2024-09-15T01:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176049+0000", 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014554100000000001 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176870+0000", 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00082102800000000001 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177006+0000", 2024-09-15T01:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013589599999999999 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177021+0000", 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5268e-05 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177364+0000", 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034309 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179998+0000", 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026343040000000001 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186627+0000", 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066287209999999997 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186678+0000", 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1044000000000001e-05 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:8 3.3s0 3:c07cd22b:::smithi17529310-3:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.175807+0000", 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.3611205420000001, 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011742891, 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 8 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175807+0000", 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175807+0000", 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175808+0000", 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5659999999999999e-06 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175816+0000", 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1770000000000001e-06 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175835+0000", 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9300000000000002e-05 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175843+0000", 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2199999999999992e-06 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177105+0000", 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012617150000000001 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177237+0000", 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013260799999999999 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177909+0000", 2024-09-15T01:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00067179900000000005 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178042+0000", 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013286099999999999 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178152+0000", 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010978499999999999 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178189+0000", 2024-09-15T01:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6859000000000001e-05 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180006+0000", 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001816757 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187405+0000", 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073992459999999999 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187550+0000", 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000144998 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:9 3.3s0 3:c07cd22b:::smithi17529310-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.175883+0000", 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.36104424, 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01603094, 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175883+0000", 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175883+0000", 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175884+0000", 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175886+0000", 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.429e-06 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175893+0000", 2024-09-15T01:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6630000000000004e-06 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175900+0000", 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999997e-06 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178335+0000", 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024349200000000001 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178422+0000", 2024-09-15T01:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7049e-05 2024-09-15T01:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187881+0000", 2024-09-15T01:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0094586740000000002 2024-09-15T01:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187983+0000", 2024-09-15T01:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102005 2024-09-15T01:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.191914+0000", 2024-09-15T01:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039315039999999997 2024-09-15T01:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:48 3.3s0 3:c4339fa1:::smithi17529310-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.188049+0000", 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.348878654, 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052579640000000004, 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 48 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188049+0000", 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188049+0000", 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188049+0000", 2024-09-15T01:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1500000000000005e-07 2024-09-15T01:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188052+0000", 2024-09-15T01:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.931e-06 2024-09-15T01:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188060+0000", 2024-09-15T01:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6639999999999998e-06 2024-09-15T01:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188064+0000", 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6449999999999998e-06 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.191955+0000", 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003891757 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.192022+0000", 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6822999999999994e-05 2024-09-15T01:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.192343+0000", 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000321155 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.192428+0000", 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5018999999999994e-05 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.193307+0000", 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00087845499999999999 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 34/31 ECSubWrite(tid=10, reqid=client.4477.0:122, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.270782+0000", 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.2661455720000001, 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062423779999999998, 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270782+0000", 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270782+0000", 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270782+0000", 2024-09-15T01:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5599999999999995e-07 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270792+0000", 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9240000000000002e-06 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276796+0000", 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060040240000000002 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276804+0000", 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2819999999999999e-06 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276840+0000", 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5920000000000002e-05 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276846+0000", 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0750000000000003e-06 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276905+0000", 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9462000000000001e-05 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276914+0000", 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4319999999999999e-06 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276945+0000", 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1334999999999997e-05 2024-09-15T01:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276959+0000", 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3730999999999999e-05 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277024+0000", 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5636999999999995e-05 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 34/31 ECSubWrite(tid=13, reqid=client.4477.0:128, at_version=34'8, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.259613554, 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.101528777, 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7999999999999995e-07 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277326+0000", 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.184e-05 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378494+0000", 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.101167374 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378498+0000", 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1150000000000004e-06 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378565+0000", 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7354999999999995e-05 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378574+0000", 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000007e-06 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378662+0000", 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8287000000000004e-05 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378672+0000", 2024-09-15T01:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3840000000000003e-06 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378705+0000", 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3506000000000001e-05 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378722+0000", 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6449000000000001e-05 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378843+0000", 2024-09-15T01:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000120937 2024-09-15T01:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:154 3.3s0 3:cf3ce551:::smithi17529310-22:head [stat,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.307955+0000", 2024-09-15T01:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.2289720420000001, 2024-09-15T01:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066651360000000003, 2024-09-15T01:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 154 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307955+0000", 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307955+0000", 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307957+0000", 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3009999999999999e-06 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307964+0000", 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9840000000000004e-06 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307976+0000", 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2626000000000001e-05 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307981+0000", 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4530000000000004e-06 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308022+0000", 2024-09-15T01:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1016999999999997e-05 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308090+0000", 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8688000000000004e-05 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308416+0000", 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032565200000000001 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308515+0000", 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8859999999999999e-05 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308523+0000", 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.554e-06 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308559+0000", 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6579999999999999e-05 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308575+0000", 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5515999999999998e-05 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.310704+0000", 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021298519999999998 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.314621+0000", 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039160530000000001 2024-09-15T01:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:39.678 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:32:39.679 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:39.676+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:39.779 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:39.778+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:39.789 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:39.789 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:39.789 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:32:39.789 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:39.798 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:32:39.880 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:39.878+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:39.936 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:39.937 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:32:39.937 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:32:39.937 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:39.945 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:32:39.988 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:39.987+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:40.045 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:40.045 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4181.0:101 1.0 e24/22 1:30306672:devicehealth::main.db.0000000000000000:head v 24'75, mlcod=23'73)", 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:17.903663+0000", 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 22.137747863000001, 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00087328700000000004, 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903663+0000", 2024-09-15T01:32:40.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903663+0000", 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903663+0000", 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.22e-07 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903729+0000", 2024-09-15T01:32:40.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6029000000000001e-05 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903753+0000", 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.41e-05 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903781+0000", 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7970000000000002e-05 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903862+0000", 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0807000000000001e-05 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.903896+0000", 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3788999999999998e-05 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904078+0000", 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000182623 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904119+0000", 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0898e-05 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904196+0000", 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7080000000000001e-05 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904496+0000", 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00029937799999999999 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:17.904536+0000", 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0391e-05 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.1s5 34/31 ECSubWrite(tid=0, reqid=client.4477.0:13, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.177809+0000", 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.8636013660000001, 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00087586900000000004, 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.051 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.177809+0000", 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.177809+0000", 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.177810+0000", 2024-09-15T01:32:40.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7500000000000001e-07 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.177899+0000", 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9914999999999995e-05 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.177951+0000", 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1384000000000001e-05 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.177957+0000", 2024-09-15T01:32:40.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4239999999999998e-06 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178003+0000", 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5846000000000001e-05 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178011+0000", 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4819999999999997e-06 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178128+0000", 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011736800000000001 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178161+0000", 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3473000000000003e-05 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178223+0000", 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1241000000000005e-05 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178249+0000", 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6118999999999998e-05 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178685+0000", 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043624199999999999 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s1 34/31 ECSubWrite(tid=0, reqid=client.4477.0:19, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.181478+0000", 2024-09-15T01:32:40.056 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.8599327240000001, 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00097614999999999996, 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181478+0000", 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181478+0000", 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181478+0000", 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0800000000000001e-07 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181537+0000", 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9324000000000003e-05 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181642+0000", 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010508700000000001 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181663+0000", 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0724999999999999e-05 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181749+0000", 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6224999999999995e-05 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181826+0000", 2024-09-15T01:32:40.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.695e-05 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181940+0000", 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011369799999999999 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181953+0000", 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3040000000000001e-05 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181997+0000", 2024-09-15T01:32:40.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.371e-05 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182424+0000", 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00042712000000000002 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182454+0000", 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0063e-05 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.061 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 34/31 ECSubWrite(tid=1, reqid=client.4477.0:34, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.187121+0000", 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.8542898299999999, 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00082887599999999998, 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187121+0000", 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187121+0000", 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187121+0000", 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5499999999999999e-07 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187277+0000", 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015635099999999999 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187318+0000", 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0802e-05 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187326+0000", 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9160000000000001e-06 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187381+0000", 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5003000000000002e-05 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187392+0000", 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0465999999999999e-05 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187534+0000", 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014225300000000001 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187547+0000", 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3016999999999999e-05 2024-09-15T01:32:40.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187616+0000", 2024-09-15T01:32:40.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9544999999999999e-05 2024-09-15T01:32:40.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187667+0000", 2024-09-15T01:32:40.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0665999999999999e-05 2024-09-15T01:32:40.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187950+0000", 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00028250200000000001 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 34/31 ECSubWrite(tid=116, reqid=client.4477.0:46, at_version=34'3, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.189743+0000", 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.8516678840000003, 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086858400000000004, 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189743+0000", 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189743+0000", 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189743+0000", 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4700000000000001e-07 2024-09-15T01:32:40.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189942+0000", 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019838200000000001 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189966+0000", 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.474e-05 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189973+0000", 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2759999999999997e-06 2024-09-15T01:32:40.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190015+0000", 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2616e-05 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190021+0000", 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4759999999999997e-06 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190143+0000", 2024-09-15T01:32:40.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012217599999999999 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190153+0000", 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6779999999999995e-06 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190220+0000", 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7181000000000006e-05 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190238+0000", 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7958e-05 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190611+0000", 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037345400000000001 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 34/31 ECSubWrite(tid=6, reqid=client.4477.0:49, at_version=34'3, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.190665+0000", 2024-09-15T01:32:40.072 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.8507458469999998, 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00080625599999999999, 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190665+0000", 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190665+0000", 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190666+0000", 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5000000000000001e-07 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190820+0000", 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000154105 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190859+0000", 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9334e-05 2024-09-15T01:32:40.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190866+0000", 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5080000000000002e-06 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190966+0000", 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010012499999999999 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190976+0000", 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0753e-05 2024-09-15T01:32:40.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.191114+0000", 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000137871 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.191144+0000", 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0139000000000001e-05 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.191210+0000", 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5743999999999999e-05 2024-09-15T01:32:40.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.191237+0000", 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6305999999999999e-05 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.191471+0000", 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023442100000000001 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 34/31 ECSubWrite(tid=0, reqid=client.4477.0:67, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.211273+0000", 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.8301370609999998, 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010066459999999999, 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211273+0000", 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211273+0000", 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211274+0000", 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6899999999999999e-07 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211568+0000", 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00029484500000000001 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211606+0000", 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7258999999999997e-05 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211615+0000", 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9600000000000006e-06 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211693+0000", 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7829000000000003e-05 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211703+0000", 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0346000000000001e-05 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211881+0000", 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000177658 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211907+0000", 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6225999999999999e-05 2024-09-15T01:32:40.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211985+0000", 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8464e-05 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212015+0000", 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0207000000000001e-05 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212280+0000", 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00026468299999999999 2024-09-15T01:32:40.082 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s1 34/31 ECSubWrite(tid=14, reqid=client.4477.0:73, at_version=34'3, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.211538+0000", 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.8298723969999999, 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000925727, 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211538+0000", 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211538+0000", 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211546+0000", 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6860000000000006e-06 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211706+0000", 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000159882 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211807+0000", 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000100975 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211815+0000", 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.816e-06 2024-09-15T01:32:40.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211889+0000", 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3831999999999994e-05 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211907+0000", 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7819e-05 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212044+0000", 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013687000000000001 2024-09-15T01:32:40.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212069+0000", 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5443999999999999e-05 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212205+0000", 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000135368 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212219+0000", 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3852999999999999e-05 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212464+0000", 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00024518199999999998 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 34/31 ECSubWrite(tid=14, reqid=client.4477.0:85, at_version=34'5, trim_to=0'0, roll_forward_to=34'3))", 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.223191+0000", 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.818219204, 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010624289999999999, 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.088 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.223191+0000", 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.223191+0000", 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.223192+0000", 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6199999999999999e-07 2024-09-15T01:32:40.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.223625+0000", 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043324299999999999 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.223646+0000", 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1073999999999998e-05 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.223653+0000", 2024-09-15T01:32:40.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1230000000000002e-06 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.223739+0000", 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6360999999999995e-05 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.223750+0000", 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0570999999999999e-05 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.223878+0000", 2024-09-15T01:32:40.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012830200000000001 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.223887+0000", 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8750000000000006e-06 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.223961+0000", 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3480999999999997e-05 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.223977+0000", 2024-09-15T01:32:40.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6650000000000002e-05 2024-09-15T01:32:40.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.224254+0000", 2024-09-15T01:32:40.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00027648700000000001 2024-09-15T01:32:40.093 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.093 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.093 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.093 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 34/31 ECSubWrite(tid=5, reqid=client.4477.0:94, at_version=34'3, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:40.093 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.233359+0000", 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.8080519119999998, 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001429119, 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.233359+0000", 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.233359+0000", 2024-09-15T01:32:40.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.233359+0000", 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0999999999999998e-07 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.234175+0000", 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00081615400000000003 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.234216+0000", 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0623000000000001e-05 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.234223+0000", 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8800000000000002e-06 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.234294+0000", 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0543999999999994e-05 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.234304+0000", 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0720000000000001e-05 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.234446+0000", 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014149900000000001 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.234452+0000", 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7259999999999996e-06 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.234535+0000", 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2553999999999996e-05 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.234600+0000", 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4500999999999998e-05 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.234788+0000", 2024-09-15T01:32:40.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018820800000000001 2024-09-15T01:32:40.100 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.100 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.100 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 34/31 ECSubWrite(tid=19, reqid=client.4477.0:97, at_version=34'7, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.235954+0000", 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.8054567170000002, 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001198099, 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.235954+0000", 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.235954+0000", 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.235954+0000", 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.82e-07 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.236542+0000", 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058734599999999998 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.236565+0000", 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3339000000000001e-05 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.236571+0000", 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4360000000000001e-06 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.236598+0000", 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7161e-05 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.236608+0000", 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0030999999999999e-05 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.236770+0000", 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000162127 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.236780+0000", 2024-09-15T01:32:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9329999999999996e-06 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.236868+0000", 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7955999999999996e-05 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.236901+0000", 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2898999999999998e-05 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.237152+0000", 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025128900000000003 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/31 ECSubWrite(tid=128, reqid=client.4477.0:100, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.240014+0000", 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.80139614, 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093151099999999999, 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240014+0000", 2024-09-15T01:32:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240014+0000", 2024-09-15T01:32:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240015+0000", 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.54e-07 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240293+0000", 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00027745800000000001 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240350+0000", 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7834999999999998e-05 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240355+0000", 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6319999999999999e-06 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240407+0000", 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2200999999999997e-05 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240412+0000", 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0679999999999996e-06 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240575+0000", 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000162292 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240583+0000", 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6850000000000007e-06 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240665+0000", 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1522999999999999e-05 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240680+0000", 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4771e-05 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240946+0000", 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000266292 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 34/31 ECSubWrite(tid=4, reqid=client.4477.0:109, at_version=34'3, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.252519+0000", 2024-09-15T01:32:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.788891837, 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011209480000000001, 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.252519+0000", 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.252519+0000", 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.252520+0000", 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2500000000000004e-07 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.252946+0000", 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00042670199999999999 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.252978+0000", 2024-09-15T01:32:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1421999999999998e-05 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.252984+0000", 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6749999999999996e-06 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.253033+0000", 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8569000000000002e-05 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.253042+0000", 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7660000000000004e-06 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.253295+0000", 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025380200000000002 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.253304+0000", 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3450000000000006e-06 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.253387+0000", 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3534000000000004e-05 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.253400+0000", 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.279e-05 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.253640+0000", 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023951800000000001 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 34/31 ECSubWrite(tid=21, reqid=client.4477.0:112, at_version=34'9, trim_to=0'0, roll_forward_to=34'7))", 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.254447+0000", 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.7869631020000001, 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012273239999999999, 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.254447+0000", 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.254447+0000", 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.254448+0000", 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1399999999999997e-07 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.255089+0000", 2024-09-15T01:32:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00064066500000000001 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.255118+0000", 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9417999999999999e-05 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.255124+0000", 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0310000000000004e-06 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.255161+0000", 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7429999999999999e-05 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.255167+0000", 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4949999999999999e-06 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.255288+0000", 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012081199999999999 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.255295+0000", 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5009999999999998e-06 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.255384+0000", 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9110999999999994e-05 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.255410+0000", 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5168e-05 2024-09-15T01:32:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.255675+0000", 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00026517899999999999 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 34/31 ECSubWrite(tid=8, reqid=client.4477.0:121, at_version=34'5, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.268618+0000", 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.7727922540000001, 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001019091, 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.268618+0000", 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.268618+0000", 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.268619+0000", 2024-09-15T01:32:40.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7300000000000005e-07 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.269025+0000", 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000405938 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.269051+0000", 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5848000000000001e-05 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.269055+0000", 2024-09-15T01:32:40.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5630000000000002e-06 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.269098+0000", 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2836000000000001e-05 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.269108+0000", 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0071000000000001e-05 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.269286+0000", 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000177226 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.269292+0000", 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.63e-06 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.269384+0000", 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1582999999999994e-05 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.269396+0000", 2024-09-15T01:32:40.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2216e-05 2024-09-15T01:32:40.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.127 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.269637+0000", 2024-09-15T01:32:40.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00024140699999999999 2024-09-15T01:32:40.127 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.127 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.127 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 34/31 ECSubWrite(tid=9, reqid=client.4477.0:124, at_version=34'5, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.271930+0000", 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.7694805659999999, 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00110957, 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.271930+0000", 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.271930+0000", 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.271931+0000", 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4799999999999998e-07 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272336+0000", 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040589199999999998 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272364+0000", 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7685999999999999e-05 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272369+0000", 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4870000000000004e-06 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272406+0000", 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7716000000000003e-05 2024-09-15T01:32:40.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272413+0000", 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2349999999999996e-06 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272544+0000", 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013174899999999999 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272552+0000", 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7179999999999996e-06 2024-09-15T01:32:40.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272664+0000", 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000111672 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272684+0000", 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0060000000000001e-05 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.273040+0000", 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035570699999999997 2024-09-15T01:32:40.132 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 34/31 ECSubWrite(tid=12, reqid=client.4477.0:127, at_version=34'7, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.276455+0000", 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.7649553190000002, 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086571700000000001, 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.133 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.276455+0000", 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.276455+0000", 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.276456+0000", 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4499999999999997e-07 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.276855+0000", 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039910500000000002 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.276879+0000", 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4343000000000001e-05 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.276889+0000", 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7820000000000005e-06 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.276923+0000", 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3501000000000003e-05 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.276928+0000", 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9790000000000001e-06 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.277011+0000", 2024-09-15T01:32:40.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3193000000000001e-05 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.277016+0000", 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5620000000000003e-06 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.277104+0000", 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8086999999999999e-05 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.277114+0000", 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8649999999999996e-06 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.277321+0000", 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020665500000000001 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/31 ECSubWrite(tid=132, reqid=client.4477.0:133, at_version=34'3, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.285357+0000", 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.7560531909999999, 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001228965, 2024-09-15T01:32:40.138 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.285357+0000", 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.285357+0000", 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.285358+0000", 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.285872+0000", 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051374400000000001 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.285896+0000", 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4241999999999999e-05 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.285903+0000", 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6069999999999996e-06 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.285941+0000", 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8071999999999997e-05 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.285946+0000", 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5280000000000003e-06 2024-09-15T01:32:40.141 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286184+0000", 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023763699999999999 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286192+0000", 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9079999999999995e-06 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286296+0000", 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000104472 2024-09-15T01:32:40.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286364+0000", 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7854999999999993e-05 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286586+0000", 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022215099999999999 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s1 34/31 ECSubWrite(tid=22, reqid=client.4477.0:139, at_version=34'5, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.294592+0000", 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.74681877, 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00098798500000000004, 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.294592+0000", 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.294592+0000", 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.294593+0000", 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5899999999999995e-07 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.294741+0000", 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014803900000000001 2024-09-15T01:32:40.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.294816+0000", 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5351000000000004e-05 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.294824+0000", 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6709999999999999e-06 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.294873+0000", 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9330999999999999e-05 2024-09-15T01:32:40.146 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.294883+0000", 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7950000000000002e-06 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.295106+0000", 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000223081 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.295116+0000", 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0020000000000001e-05 2024-09-15T01:32:40.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.295234+0000", 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118237 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.295254+0000", 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9862999999999999e-05 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.295580+0000", 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032583800000000001 2024-09-15T01:32:40.148 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 34/31 ECSubWrite(tid=136, reqid=client.4477.0:154, at_version=34'9, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.308440+0000", 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.7329703190000001, 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00078582100000000002, 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.308440+0000", 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.308440+0000", 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.308441+0000", 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4199999999999995e-07 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.308459+0000", 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8487000000000001e-05 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.308486+0000", 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6149999999999999e-05 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.308493+0000", 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9720000000000003e-06 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.308530+0000", 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7134000000000001e-05 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.308536+0000", 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7340000000000002e-06 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.308589+0000", 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2978e-05 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.308596+0000", 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4620000000000001e-06 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.308622+0000", 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6517999999999999e-05 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.308632+0000", 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9930000000000007e-06 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.154 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.309226+0000", 2024-09-15T01:32:40.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00059365100000000005 2024-09-15T01:32:40.154 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.154 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.154 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.154 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.154 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.154 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:40.154 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:32:40.155 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:40.098+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:40.196 INFO:tasks.rados.rados.0.smithi175.stdout:54: done (5 left) 2024-09-15T01:32:40.196 INFO:tasks.rados.rados.0.smithi175.stdout:59: done (4 left) 2024-09-15T01:32:40.196 INFO:tasks.rados.rados.0.smithi175.stdout:64: done (3 left) 2024-09-15T01:32:40.196 INFO:tasks.rados.rados.0.smithi175.stdout:65: done (2 left) 2024-09-15T01:32:40.196 INFO:tasks.rados.rados.0.smithi175.stdout:66: done (1 left) 2024-09-15T01:32:40.196 INFO:tasks.rados.rados.0.smithi175.stdout:67: done (0 left) 2024-09-15T01:32:40.196 INFO:tasks.rados.rados.0.smithi175.stdout:68: delete oid 27 current snap is 3 2024-09-15T01:32:40.197 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-15T01:32:40.200 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:40.198+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:40.203 INFO:tasks.rados.rados.0.smithi175.stdout:68: done (0 left) 2024-09-15T01:32:40.203 INFO:tasks.rados.rados.0.smithi175.stdout:69: rollback oid 30 current snap is 3 2024-09-15T01:32:40.204 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 30 to 1 2024-09-15T01:32:40.204 INFO:tasks.rados.rados.0.smithi175.stdout:70: rollback oid 29 current snap is 3 2024-09-15T01:32:40.204 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 29 to 1 2024-09-15T01:32:40.204 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 14 current snap is 3 2024-09-15T01:32:40.204 INFO:tasks.rados.rados.0.smithi175.stdout:71: seq_num 56 ranges {229376=573440,802816=376832} 2024-09-15T01:32:40.208 INFO:tasks.rados.rados.0.smithi175.stdout:71: writing smithi17529310-14 from 229376 to 802816 tid 1 2024-09-15T01:32:40.211 INFO:tasks.rados.rados.0.smithi175.stdout:71: writing smithi17529310-14 from 802816 to 1179648 tid 2 2024-09-15T01:32:40.211 INFO:tasks.rados.rados.0.smithi175.stdout:70: finishing rollback tid 0 to smithi17529310-29 2024-09-15T01:32:40.211 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 32 current snap is 3 2024-09-15T01:32:40.211 INFO:tasks.rados.rados.0.smithi175.stdout:70: finishing rollback tid 1 to smithi17529310-29 2024-09-15T01:32:40.211 INFO:tasks.rados.rados.0.smithi175.stdout:72: seq_num 57 ranges {524288=589824,1114112=376832} 2024-09-15T01:32:40.217 INFO:tasks.rados.rados.0.smithi175.stdout:72: writing smithi17529310-32 from 524288 to 1114112 tid 1 2024-09-15T01:32:40.219 INFO:tasks.rados.rados.0.smithi175.stdout:72: writing smithi17529310-32 from 1114112 to 1490944 tid 2 2024-09-15T01:32:40.219 INFO:tasks.rados.rados.0.smithi175.stdout:73: snap_remove snap 2 2024-09-15T01:32:40.259 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:40.259 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:40.259 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:32:40.259 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:40.268 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:32:40.301 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:40.300+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:40.364 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:32:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:32:40.365 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:40.373 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:32:40.401 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:40.400+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:40.474 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:40.474 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:32:40.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:32:40.474 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:32:40.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.474 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.5s6 33/31 ECSubWrite(tid=1, reqid=client.4477.0:11, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.474 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.178219+0000", 2024-09-15T01:32:40.474 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.2933169260000001, 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001342698, 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178219+0000", 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178219+0000", 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178219+0000", 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.39e-07 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178226+0000", 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7959999999999998e-06 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178292+0000", 2024-09-15T01:32:40.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6526000000000006e-05 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178297+0000", 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1569999999999999e-06 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178516+0000", 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021854399999999999 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178524+0000", 2024-09-15T01:32:40.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8949999999999998e-06 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178615+0000", 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1028999999999999e-05 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178621+0000", 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1820000000000003e-06 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178646+0000", 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5007999999999998e-05 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178676+0000", 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0334000000000001e-05 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179561+0000", 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088488800000000002 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 34/31 ECSubWrite(tid=1, reqid=client.4477.0:28, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.178351+0000", 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.2931843919999997, 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010995930000000001, 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178351+0000", 2024-09-15T01:32:40.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178351+0000", 2024-09-15T01:32:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178351+0000", 2024-09-15T01:32:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1e-07 2024-09-15T01:32:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178420+0000", 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8765999999999999e-05 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178461+0000", 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0676000000000002e-05 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178467+0000", 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4670000000000001e-06 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178532+0000", 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4482999999999999e-05 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178539+0000", 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3339999999999996e-06 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178689+0000", 2024-09-15T01:32:40.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015008900000000001 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178701+0000", 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1460999999999999e-05 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178746+0000", 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5358000000000002e-05 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.178756+0000", 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0217000000000001e-05 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179451+0000", 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00069453199999999998 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.485 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s5 34/31 ECSubWrite(tid=0, reqid=client.4477.0:19, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.181875+0000", 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.2896605540000001, 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001217299, 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181875+0000", 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181875+0000", 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181875+0000", 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3300000000000001e-07 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181921+0000", 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6286999999999999e-05 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181971+0000", 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9002000000000003e-05 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181985+0000", 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4389000000000001e-05 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182071+0000", 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6222000000000002e-05 2024-09-15T01:32:40.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182080+0000", 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5150000000000006e-06 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182207+0000", 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000127384 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182752+0000", 2024-09-15T01:32:40.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00054473100000000001 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182808+0000", 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6657999999999999e-05 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182850+0000", 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1613000000000003e-05 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183092+0000", 2024-09-15T01:32:40.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000242265 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 34/31 ECSubWrite(tid=3, reqid=client.4477.0:37, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.183114+0000", 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.288421821, 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00097461900000000003, 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183114+0000", 2024-09-15T01:32:40.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183114+0000", 2024-09-15T01:32:40.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183114+0000", 2024-09-15T01:32:40.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5900000000000003e-07 2024-09-15T01:32:40.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183282+0000", 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016760599999999999 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183326+0000", 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3900999999999999e-05 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183333+0000", 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0260000000000002e-06 2024-09-15T01:32:40.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183402+0000", 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9669e-05 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183413+0000", 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0436e-05 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183655+0000", 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00024221799999999999 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183666+0000", 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1053000000000001e-05 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183762+0000", 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6052999999999997e-05 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183778+0000", 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5613000000000001e-05 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184088+0000", 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000310685 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 34/31 ECSubWrite(tid=1, reqid=client.4477.0:34, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.187278+0000", 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.2842578790000001, 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092075299999999996, 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187278+0000", 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187278+0000", 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187278+0000", 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4400000000000001e-07 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187379+0000", 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010080400000000001 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187398+0000", 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9451999999999999e-05 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187404+0000", 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2539999999999998e-06 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187460+0000", 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5686999999999997e-05 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187470+0000", 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7070000000000004e-06 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187680+0000", 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000210652 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187738+0000", 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7496e-05 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187802+0000", 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4109000000000006e-05 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187832+0000", 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9810000000000001e-05 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.188198+0000", 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00036653800000000002 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 34/31 ECSubWrite(tid=6, reqid=client.4477.0:43, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.501 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.187534+0000", 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.2840014420000001, 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00081715399999999995, 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187534+0000", 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187534+0000", 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187535+0000", 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1200000000000003e-07 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187674+0000", 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000139696 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187707+0000", 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2512999999999999e-05 2024-09-15T01:32:40.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187713+0000", 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.207e-06 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187769+0000", 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5792999999999999e-05 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187777+0000", 2024-09-15T01:32:40.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5499999999999995e-06 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187928+0000", 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015101599999999999 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187953+0000", 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4592000000000001e-05 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.188016+0000", 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2586e-05 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.188034+0000", 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8542e-05 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.188351+0000", 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00031704699999999999 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.506 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 34/31 ECSubWrite(tid=10, reqid=client.4477.0:58, at_version=34'3, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.197163+0000", 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.2743722389999999, 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088844399999999995, 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.197163+0000", 2024-09-15T01:32:40.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.197163+0000", 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.197164+0000", 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1199999999999999e-07 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.197544+0000", 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000380181 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.197579+0000", 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5250000000000003e-05 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.197590+0000", 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0861e-05 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.197627+0000", 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7119999999999997e-05 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.197633+0000", 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8889999999999999e-06 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.197733+0000", 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000100282 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.197740+0000", 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8800000000000002e-06 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.197801+0000", 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1079000000000003e-05 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.197813+0000", 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1810999999999999e-05 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.198052+0000", 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000238879 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.2s3 34/31 ECSubWrite(tid=128, reqid=client.4477.0:100, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.239998+0000", 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.2315372959999999, 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011981660000000001, 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.512 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.239998+0000", 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.239998+0000", 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.239998+0000", 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.48e-07 2024-09-15T01:32:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240556+0000", 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00055780999999999995 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240588+0000", 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1795999999999998e-05 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240594+0000", 2024-09-15T01:32:40.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3600000000000001e-06 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240648+0000", 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4104999999999998e-05 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240659+0000", 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0083e-05 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240840+0000", 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181113 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240905+0000", 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5430000000000002e-05 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240983+0000", 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8285e-05 2024-09-15T01:32:40.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.240999+0000", 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6002999999999999e-05 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.241196+0000", 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000197033 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.5s6 34/31 ECSubWrite(tid=7, reqid=client.4477.0:103, at_version=34'5, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.243418+0000", 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.2281176020000002, 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00098405400000000005, 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.243418+0000", 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.243418+0000", 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.243419+0000", 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0100000000000004e-07 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.243731+0000", 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00031250700000000001 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.243758+0000", 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6678000000000001e-05 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.243763+0000", 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7979999999999996e-06 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.243811+0000", 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8179000000000001e-05 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.243821+0000", 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0011e-05 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.244080+0000", 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000258955 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.244089+0000", 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4739999999999994e-06 2024-09-15T01:32:40.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.244167+0000", 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7367999999999995e-05 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.244178+0000", 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1647e-05 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.244402+0000", 2024-09-15T01:32:40.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022363600000000001 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as5 34/31 ECSubWrite(tid=6, reqid=client.4477.0:118, at_version=34'3, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.262980+0000", 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.2085558949999999, 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001062557, 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.262980+0000", 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.262980+0000", 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.262981+0000", 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8029999999999999e-06 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263374+0000", 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039269499999999998 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263409+0000", 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4520000000000002e-05 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263416+0000", 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7770000000000002e-06 2024-09-15T01:32:40.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263467+0000", 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0399000000000003e-05 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263478+0000", 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1043000000000001e-05 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263596+0000", 2024-09-15T01:32:40.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118386 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263605+0000", 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2919999999999996e-06 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263699+0000", 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4189000000000002e-05 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263863+0000", 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000164365 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.264042+0000", 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017908799999999999 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 34/31 ECSubWrite(tid=9, reqid=client.4477.0:124, at_version=34'5, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:40.528 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.272105+0000", 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.1994300869999996, 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010842670000000001, 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272105+0000", 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272105+0000", 2024-09-15T01:32:40.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272106+0000", 2024-09-15T01:32:40.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6599999999999996e-07 2024-09-15T01:32:40.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272503+0000", 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039722999999999999 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272531+0000", 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7844e-05 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272536+0000", 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2800000000000003e-06 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272581+0000", 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4975999999999998e-05 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272587+0000", 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.694e-06 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272748+0000", 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016125199999999999 2024-09-15T01:32:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272755+0000", 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8299999999999998e-06 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272852+0000", 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6988e-05 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.272896+0000", 2024-09-15T01:32:40.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4039999999999998e-05 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.273190+0000", 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000293467 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.2s3 34/31 ECSubWrite(tid=132, reqid=client.4477.0:133, at_version=34'3, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.285528+0000", 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.1860078500000002, 2024-09-15T01:32:40.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001192982, 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.285528+0000", 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.285528+0000", 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.285528+0000", 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8899999999999999e-07 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286035+0000", 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050683 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286088+0000", 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2775999999999998e-05 2024-09-15T01:32:40.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286094+0000", 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8749999999999997e-06 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286153+0000", 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9038e-05 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286162+0000", 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2180000000000005e-06 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286356+0000", 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000193599 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286364+0000", 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0560000000000005e-06 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286461+0000", 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6930999999999997e-05 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286502+0000", 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1513e-05 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.286721+0000", 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000218457 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.539 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.540 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 34/31 ECSubWrite(tid=24, reqid=client.4477.0:136, at_version=34'5, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:40.540 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.289193+0000", 2024-09-15T01:32:40.540 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.1823423110000002, 2024-09-15T01:32:40.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0013471620000000001, 2024-09-15T01:32:40.540 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.540 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.540 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289193+0000", 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289193+0000", 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289194+0000", 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4700000000000002e-07 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289961+0000", 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076762 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289983+0000", 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1611e-05 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289988+0000", 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4700000000000001e-06 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.290026+0000", 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7681000000000001e-05 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.290036+0000", 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8449999999999998e-06 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.290190+0000", 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000153609 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.290197+0000", 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5950000000000001e-06 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.290294+0000", 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7139000000000006e-05 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.290340+0000", 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5361000000000002e-05 2024-09-15T01:32:40.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.290540+0000", 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020078399999999999 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 34/31 ECSubWrite(tid=30, reqid=client.4477.0:142, at_version=34'7, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.296439+0000", 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.1750966710000004, 2024-09-15T01:32:40.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000994263, 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.296439+0000", 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.296439+0000", 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.546 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.296439+0000", 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6400000000000003e-07 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.296968+0000", 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000528383 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.296993+0000", 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5451000000000001e-05 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.296997+0000", 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2749999999999997e-06 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.297030+0000", 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2150000000000002e-05 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.297036+0000", 2024-09-15T01:32:40.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.46e-06 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.297161+0000", 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000124511 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.297168+0000", 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7980000000000006e-06 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.297234+0000", 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5697999999999993e-05 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.297245+0000", 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.297433+0000", 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018795300000000001 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.550 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 37/36 ECSubWrite(tid=30, reqid=client.4477.0:185, at_version=37'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:39.189067+0000", 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 1.2824683160000001, 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0052050109999999998, 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.189067+0000", 2024-09-15T01:32:40.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.189067+0000", 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.189068+0000", 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3900000000000004e-07 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.189095+0000", 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6579000000000001e-05 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.189124+0000", 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9615999999999999e-05 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.189133+0000", 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9819999999999997e-06 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.189186+0000", 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3161999999999997e-05 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.189196+0000", 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9199999999999999e-06 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.189430+0000", 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023374299999999999 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.189448+0000", 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8238999999999999e-05 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.189486+0000", 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7750999999999998e-05 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.189504+0000", 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8522e-05 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194272+0000", 2024-09-15T01:32:40.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0047677580000000004 2024-09-15T01:32:40.556 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.556 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.556 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.556 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 37/36 ECSubWrite(tid=33, reqid=client.4477.0:185, at_version=0'0, trim_to=0'0, roll_forward_to=37'7))", 2024-09-15T01:32:40.556 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:39.191796+0000", 2024-09-15T01:32:40.556 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 1.279739325, 2024-09-15T01:32:40.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0026716639999999998, 2024-09-15T01:32:40.556 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.191796+0000", 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.191796+0000", 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.191797+0000", 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2500000000000004e-07 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.191812+0000", 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4859e-05 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.191840+0000", 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8084000000000001e-05 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.191848+0000", 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6079999999999998e-06 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.191896+0000", 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8093999999999997e-05 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.191908+0000", 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1846e-05 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.192015+0000", 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107457 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.192026+0000", 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.099e-05 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193468+0000", 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0014415840000000001 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193497+0000", 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9646e-05 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194468+0000", 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00097057100000000004 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as5 37/36 ECSubWrite(tid=12, reqid=client.4477.0:192, at_version=37'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.561 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:39.201904+0000", 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 1.2696311149999999, 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0024654059999999998, 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.201904+0000", 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.201904+0000", 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.201905+0000", 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8999999999999995e-07 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202379+0000", 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000473327 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202415+0000", 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6063999999999999e-05 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202421+0000", 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5740000000000001e-06 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202483+0000", 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1654000000000003e-05 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202497+0000", 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4449e-05 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202766+0000", 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00026823899999999998 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202777+0000", 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1695999999999999e-05 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202886+0000", 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108621 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202906+0000", 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9779000000000002e-05 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.204370+0000", 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001464213 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.566 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 38/31 ECSubWrite(tid=15, reqid=client.4477.0:206, at_version=38'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:40.219291+0000", 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.25224492599999998, 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0042530989999999998, 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.219291+0000", 2024-09-15T01:32:40.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.219291+0000", 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.219291+0000", 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8200000000000001e-07 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.219971+0000", 2024-09-15T01:32:40.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00067996099999999998 2024-09-15T01:32:40.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.219993+0000", 2024-09-15T01:32:40.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2079e-05 2024-09-15T01:32:40.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.220001+0000", 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2390000000000002e-06 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.220051+0000", 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9712000000000001e-05 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.220062+0000", 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.081e-05 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.220335+0000", 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00027356800000000002 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.220342+0000", 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.109e-06 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.220427+0000", 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4567999999999994e-05 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.220443+0000", 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6095000000000001e-05 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.223544+0000", 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0031006760000000001 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.572 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 38/31 ECSubWrite(tid=16, reqid=client.4477.0:207, at_version=38'11, trim_to=0'0, roll_forward_to=36'8))", 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:40.220777+0000", 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.250758437, 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003122437, 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.220777+0000", 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.220777+0000", 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.220778+0000", 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1500000000000003e-07 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221087+0000", 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00030949100000000001 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221123+0000", 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5287000000000003e-05 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221129+0000", 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5370000000000003e-06 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221182+0000", 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2352000000000003e-05 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221191+0000", 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9619999999999999e-06 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221340+0000", 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014941800000000001 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221350+0000", 2024-09-15T01:32:40.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7359999999999996e-06 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221426+0000", 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5924e-05 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221450+0000", 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4856000000000001e-05 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.223900+0000", 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.002449059 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 38/31 ECSubWrite(tid=17, reqid=client.4477.0:208, at_version=38'12, trim_to=0'0, roll_forward_to=36'8))", 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:40.221376+0000", 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.25015980999999998, 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0029150999999999999, 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221376+0000", 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221376+0000", 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221376+0000", 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9699999999999996e-07 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221392+0000", 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5889e-05 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221423+0000", 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0689999999999999e-05 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221429+0000", 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8980000000000001e-06 2024-09-15T01:32:40.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221637+0000", 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000207925 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221644+0000", 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6790000000000005e-06 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221730+0000", 2024-09-15T01:32:40.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5439999999999995e-05 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221737+0000", 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2439999999999996e-06 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221783+0000", 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5748999999999999e-05 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.221807+0000", 2024-09-15T01:32:40.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4729e-05 2024-09-15T01:32:40.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.224291+0000", 2024-09-15T01:32:40.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00248336 2024-09-15T01:32:40.583 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.583 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.583 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.583 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.583 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.583 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:40.585 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:32:40.585 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:40.583+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:40.685 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:40.683+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:40.723 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:40.723 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:40.723 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:32:40.723 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:40.731 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:32:40.786 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:40.784+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:40.868 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:40.868 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:40.868 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:32:40.868 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:32:40.868 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:40.876 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:32:40.886 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:40.885+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:40.975 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:40.975 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:32:40.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:32:40.975 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:32:40.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.1s4 34/31 ECSubWrite(tid=0, reqid=client.4477.0:13, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.179090+0000", 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.7931271940000002, 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00108424, 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179090+0000", 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179090+0000", 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179090+0000", 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8000000000000002e-07 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179182+0000", 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2386000000000006e-05 2024-09-15T01:32:40.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179234+0000", 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1999000000000001e-05 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179242+0000", 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1699999999999997e-06 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179292+0000", 2024-09-15T01:32:40.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9997999999999998e-05 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179301+0000", 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.422e-06 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179407+0000", 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000105851 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179554+0000", 2024-09-15T01:32:40.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014693599999999999 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179608+0000", 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4061999999999998e-05 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179637+0000", 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9297000000000001e-05 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180174+0000", 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000536839 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.1s4 34/31 ECSubWrite(tid=2, reqid=client.4477.0:14, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.179186+0000", 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.7930312390000003, 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0014413780000000001, 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179186+0000", 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179186+0000", 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179186+0000", 2024-09-15T01:32:40.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179197+0000", 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0847e-05 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179347+0000", 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000150452 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179363+0000", 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5065000000000001e-05 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179444+0000", 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1753000000000001e-05 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179452+0000", 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4900000000000003e-06 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179534+0000", 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2298000000000004e-05 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179555+0000", 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0628e-05 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179609+0000", 2024-09-15T01:32:40.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4085000000000001e-05 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.179670+0000", 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1525000000000005e-05 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180627+0000", 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000956693 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.1s4 34/31 ECSubWrite(tid=4, reqid=client.4477.0:31, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.181381+0000", 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.790835918, 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011871620000000001, 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181381+0000", 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181381+0000", 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181382+0000", 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7899999999999997e-07 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181752+0000", 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000370338 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181804+0000", 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1950000000000002e-05 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181812+0000", 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0190000000000006e-06 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181863+0000", 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1085000000000003e-05 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181873+0000", 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5219999999999996e-06 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182003+0000", 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000129775 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182141+0000", 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000138085 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182231+0000", 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0657999999999998e-05 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182535+0000", 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00030367700000000003 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182568+0000", 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3173999999999998e-05 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.991 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.992 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.992 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.9s6 34/31 ECSubWrite(tid=1, reqid=client.4477.0:22, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.992 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.181647+0000", 2024-09-15T01:32:40.992 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.7905699799999999, 2024-09-15T01:32:40.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010622769999999999, 2024-09-15T01:32:40.992 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.992 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.992 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181647+0000", 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181647+0000", 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181647+0000", 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.16e-07 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181762+0000", 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011540300000000001 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181853+0000", 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0735000000000005e-05 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181860+0000", 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6420000000000001e-06 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182044+0000", 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018416 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182051+0000", 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9539999999999998e-06 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182198+0000", 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000146859 2024-09-15T01:32:40.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182233+0000", 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5510999999999999e-05 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182279+0000", 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5834000000000002e-05 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:40.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182674+0000", 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039441500000000001 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182709+0000", 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5547999999999999e-05 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 34/31 ECSubWrite(tid=3, reqid=client.4477.0:37, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:40.997 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.183147+0000", 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.7890703170000002, 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010473380000000001, 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183147+0000", 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183147+0000", 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:40.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183147+0000", 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4799999999999998e-07 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183495+0000", 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000347955 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183547+0000", 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2221e-05 2024-09-15T01:32:40.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183554+0000", 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.511e-06 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183641+0000", 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6965000000000005e-05 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183649+0000", 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0430000000000008e-06 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183786+0000", 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013710999999999999 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183829+0000", 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3294999999999998e-05 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183891+0000", 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1523e-05 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183910+0000", 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9579e-05 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184194+0000", 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00028358799999999998 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as6 34/31 ECSubWrite(tid=6, reqid=client.4477.0:118, at_version=34'3, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.263092+0000", 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.7091248270000001, 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011956149999999999, 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263092+0000", 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263092+0000", 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263094+0000", 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5600000000000001e-06 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263675+0000", 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058139800000000001 2024-09-15T01:32:41.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263724+0000", 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8619999999999999e-05 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263730+0000", 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263794+0000", 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4172999999999997e-05 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263804+0000", 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2930000000000006e-06 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263894+0000", 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0586000000000003e-05 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263902+0000", 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8990000000000001e-06 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.263997+0000", 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5006000000000005e-05 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.264025+0000", 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7359999999999999e-05 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.264288+0000", 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000263028 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 34/31 ECSubWrite(tid=24, reqid=client.4477.0:136, at_version=34'5, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.289226+0000", 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 4.6829910180000001, 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011010480000000001, 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.008 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289226+0000", 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289226+0000", 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289227+0000", 2024-09-15T01:32:41.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5099999999999999e-07 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289747+0000", 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052074 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289772+0000", 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4720999999999999e-05 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289777+0000", 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5310000000000003e-06 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289813+0000", 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6222e-05 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289818+0000", 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6949999999999997e-06 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289906+0000", 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7262999999999994e-05 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.289915+0000", 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5750000000000007e-06 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.290011+0000", 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5422999999999998e-05 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.290023+0000", 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2678e-05 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.290327+0000", 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00030344900000000001 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 37/31 ECSubWrite(tid=19, reqid=client.4477.0:186, at_version=37'12, trim_to=0'0, roll_forward_to=34'10))", 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:39.193411+0000", 2024-09-15T01:32:41.013 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 1.7788060670000001, 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001526985, 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193411+0000", 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193411+0000", 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193411+0000", 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2200000000000001e-07 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193429+0000", 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8258e-05 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193460+0000", 2024-09-15T01:32:41.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1210000000000001e-05 2024-09-15T01:32:41.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193468+0000", 2024-09-15T01:32:41.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.418e-06 2024-09-15T01:32:41.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193538+0000", 2024-09-15T01:32:41.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0222999999999995e-05 2024-09-15T01:32:41.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193550+0000", 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1603999999999999e-05 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193818+0000", 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00026859699999999998 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193831+0000", 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2843999999999999e-05 2024-09-15T01:32:41.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193881+0000", 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9500999999999999e-05 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.193902+0000", 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1314e-05 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194938+0000", 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010358940000000001 2024-09-15T01:32:41.018 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 37/31 ECSubWrite(tid=20, reqid=client.4477.0:187, at_version=37'13, trim_to=0'0, roll_forward_to=34'10))", 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:39.194446+0000", 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 1.7777709210000001, 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001673906, 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194446+0000", 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194446+0000", 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194447+0000", 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8599999999999997e-07 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194466+0000", 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9414000000000001e-05 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194500+0000", 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3821e-05 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194507+0000", 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9129999999999997e-06 2024-09-15T01:32:41.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194569+0000", 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2100999999999994e-05 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194589+0000", 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9796000000000002e-05 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194685+0000", 2024-09-15T01:32:41.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6459000000000006e-05 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194694+0000", 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1509999999999993e-06 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194722+0000", 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7725e-05 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.194740+0000", 2024-09-15T01:32:41.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7873999999999999e-05 2024-09-15T01:32:41.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.196120+0000", 2024-09-15T01:32:41.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001379866 2024-09-15T01:32:41.024 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.024 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.024 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.025 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 37/31 ECSubWrite(tid=21, reqid=client.4477.0:188, at_version=37'14, trim_to=0'0, roll_forward_to=34'10))", 2024-09-15T01:32:41.025 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:39.195204+0000", 2024-09-15T01:32:41.025 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 1.777012587, 2024-09-15T01:32:41.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001034821, 2024-09-15T01:32:41.025 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.025 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.025 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.195204+0000", 2024-09-15T01:32:41.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.195204+0000", 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.195205+0000", 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2399999999999997e-07 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.195228+0000", 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2945000000000001e-05 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.195265+0000", 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6591999999999998e-05 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.195271+0000", 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4110000000000001e-06 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.195324+0000", 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2639999999999997e-05 2024-09-15T01:32:41.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.195333+0000", 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4280000000000002e-06 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.195422+0000", 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9118999999999998e-05 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.195431+0000", 2024-09-15T01:32:41.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6580000000000008e-06 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.195478+0000", 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6662000000000001e-05 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.195492+0000", 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4268e-05 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.196239+0000", 2024-09-15T01:32:41.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00074727400000000003 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as6 37/36 ECSubWrite(tid=12, reqid=client.4477.0:192, at_version=37'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:39.201994+0000", 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 1.7702233540000001, 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003031195, 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.201994+0000", 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.201994+0000", 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.201994+0000", 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9400000000000004e-07 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202519+0000", 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052463100000000001 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202555+0000", 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6013999999999998e-05 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202562+0000", 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1999999999999997e-06 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202600+0000", 2024-09-15T01:32:41.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7645999999999998e-05 2024-09-15T01:32:41.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202608+0000", 2024-09-15T01:32:41.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1130000000000001e-06 2024-09-15T01:32:41.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202913+0000", 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00030474199999999999 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.202923+0000", 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7969999999999995e-06 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.203012+0000", 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9671000000000003e-05 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.203026+0000", 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4239999999999999e-05 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.205025+0000", 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0019982469999999999 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as6 37/36 ECSubWrite(tid=13, reqid=client.4477.0:192, at_version=0'0, trim_to=0'0, roll_forward_to=37'6))", 2024-09-15T01:32:41.035 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:39.203783+0000", 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 1.768434168, 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001372023, 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.203783+0000", 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.203783+0000", 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.203784+0000", 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5499999999999997e-07 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.203808+0000", 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4020999999999999e-05 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.203846+0000", 2024-09-15T01:32:41.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8577e-05 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.203853+0000", 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1189999999999999e-06 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.203903+0000", 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0093999999999998e-05 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.203914+0000", 2024-09-15T01:32:41.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0584000000000001e-05 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.204046+0000", 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000131874 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.204055+0000", 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2080000000000006e-06 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.204096+0000", 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1251999999999997e-05 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.204109+0000", 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2849e-05 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:39.205155+0000", 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010455900000000001 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 38/31 ECSubWrite(tid=23, reqid=client.4477.0:195, at_version=38'16, trim_to=0'0, roll_forward_to=37'14))", 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:40.197661+0000", 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.77455598999999997, 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015447970000000001, 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.197661+0000", 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.197661+0000", 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.197661+0000", 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5199999999999998e-07 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.197691+0000", 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9913e-05 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.197730+0000", 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9298000000000002e-05 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.197738+0000", 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6620000000000005e-06 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.197800+0000", 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1826000000000001e-05 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.197809+0000", 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6549999999999999e-06 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.198006+0000", 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019689499999999999 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.198012+0000", 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2660000000000001e-06 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.198047+0000", 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4929999999999999e-05 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.198063+0000", 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6066000000000002e-05 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.199206+0000", 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001143034 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.045 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 38/31 ECSubWrite(tid=50, reqid=client.4477.0:199, at_version=38'15, trim_to=0'0, roll_forward_to=34'13))", 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:40.205217+0000", 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.76699957600000002, 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0042686449999999997, 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.205217+0000", 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.205217+0000", 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.205217+0000", 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5200000000000001e-07 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.205232+0000", 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.47e-05 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.205258+0000", 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5551999999999999e-05 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.205266+0000", 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0369999999999995e-06 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.205305+0000", 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9502999999999998e-05 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.205314+0000", 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6100000000000006e-06 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.205560+0000", 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00024573599999999998 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.205568+0000", 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3960000000000006e-06 2024-09-15T01:32:41.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.205609+0000", 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1223000000000002e-05 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.205623+0000", 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3961000000000001e-05 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.209486+0000", 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0038627750000000001 2024-09-15T01:32:41.050 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 38/31 ECSubWrite(tid=51, reqid=client.4477.0:200, at_version=38'16, trim_to=0'0, roll_forward_to=34'13))", 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:40.206025+0000", 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.76619233099999995, 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0066684200000000004, 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206025+0000", 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206025+0000", 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206025+0000", 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6000000000000003e-07 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206043+0000", 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8005000000000001e-05 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206073+0000", 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9357e-05 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206079+0000", 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0240000000000003e-06 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206120+0000", 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1343e-05 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206128+0000", 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9610000000000005e-06 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206207+0000", 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9270000000000005e-05 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206242+0000", 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5034999999999999e-05 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206277+0000", 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4394000000000003e-05 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206301+0000", 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.478e-05 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212693+0000", 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.006391491 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 38/31 ECSubWrite(tid=52, reqid=client.4477.0:201, at_version=38'17, trim_to=0'0, roll_forward_to=34'13))", 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:40.206780+0000", 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.76543675, 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0060500090000000003, 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.056 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206780+0000", 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206780+0000", 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206783+0000", 2024-09-15T01:32:41.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2610000000000002e-06 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206809+0000", 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5910000000000001e-05 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206840+0000", 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0636999999999998e-05 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206847+0000", 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6189999999999996e-06 2024-09-15T01:32:41.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206894+0000", 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7611999999999997e-05 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206899+0000", 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2089999999999996e-06 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206972+0000", 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2611e-05 2024-09-15T01:32:41.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.206979+0000", 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4309999999999997e-06 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.207012+0000", 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.286e-05 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.207023+0000", 2024-09-15T01:32:41.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1075e-05 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212830+0000", 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0058067839999999997 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 38/31 ECSubWrite(tid=53, reqid=client.4477.0:201, at_version=0'0, trim_to=0'0, roll_forward_to=38'17))", 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:40.208143+0000", 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.764074217, 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0048120999999999997, 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.061 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.208143+0000", 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.208143+0000", 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.208143+0000", 2024-09-15T01:32:41.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1500000000000005e-07 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.208151+0000", 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3089999999999996e-06 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.208173+0000", 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1656999999999999e-05 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.208179+0000", 2024-09-15T01:32:41.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5119999999999999e-06 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.208226+0000", 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7349999999999999e-05 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.208235+0000", 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3980000000000005e-06 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.208351+0000", 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011516299999999999 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.208363+0000", 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2755000000000001e-05 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.208390+0000", 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6591e-05 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.208399+0000", 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2220000000000008e-06 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212955+0000", 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045556279999999999 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.9s6 38/31 ECSubWrite(tid=26, reqid=client.4477.0:202, at_version=38'10, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:32:41.066 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:40.210084+0000", 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.76213299099999998, 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0038321539999999999, 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.210084+0000", 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.210084+0000", 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.210084+0000", 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2600000000000001e-07 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.210186+0000", 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010239900000000001 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.210234+0000", 2024-09-15T01:32:41.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7398999999999998e-05 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.210243+0000", 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6600000000000001e-06 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.210302+0000", 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9598000000000001e-05 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.210312+0000", 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3859999999999996e-06 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.210721+0000", 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040905299999999999 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.210731+0000", 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0417999999999999e-05 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.210822+0000", 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1160000000000001e-05 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.210839+0000", 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7098000000000001e-05 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.213916+0000", 2024-09-15T01:32:41.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030767569999999998 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.9s6 38/31 ECSubWrite(tid=27, reqid=client.4477.0:203, at_version=38'11, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:40.211519+0000", 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.76069830500000002, 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025491799999999998, 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.211519+0000", 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.211519+0000", 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.211519+0000", 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5499999999999997e-07 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.211559+0000", 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9382999999999999e-05 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.211597+0000", 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8296000000000001e-05 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.211604+0000", 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4540000000000004e-06 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.211646+0000", 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2889000000000002e-05 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.211655+0000", 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1100000000000003e-06 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.211737+0000", 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2225000000000006e-05 2024-09-15T01:32:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.211742+0000", 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2730000000000002e-06 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212406+0000", 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066407 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212420+0000", 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4358e-05 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.214068+0000", 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001647267 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.9s6 38/31 ECSubWrite(tid=28, reqid=client.4477.0:204, at_version=38'12, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:40.212178+0000", 2024-09-15T01:32:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.76003928300000001, 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0020413850000000002, 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212178+0000", 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212178+0000", 2024-09-15T01:32:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212178+0000", 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1399999999999997e-07 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212191+0000", 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2947e-05 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212217+0000", 2024-09-15T01:32:41.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5843999999999999e-05 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212223+0000", 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.784e-06 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212278+0000", 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4839000000000001e-05 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212287+0000", 2024-09-15T01:32:41.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8900000000000002e-06 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212382+0000", 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4335999999999999e-05 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212409+0000", 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7444e-05 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212445+0000", 2024-09-15T01:32:41.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6127000000000002e-05 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.212459+0000", 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3779e-05 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:40.214219+0000", 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001759881 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.082 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.083 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:41.084 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-15T01:32:41.085 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:41.082+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:41.182 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:32:41.182 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:41.184 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:41.183+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:41.190 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-15T01:32:41.199 INFO:tasks.rados.rados.0.smithi175.stdout:73: done (4 left) 2024-09-15T01:32:41.199 INFO:tasks.rados.rados.0.smithi175.stdout:74: copy_from oid 6 from oid 22 current snap is 3 2024-09-15T01:32:41.199 INFO:tasks.rados.rados.0.smithi175.stdout:70: finishing rollback tid 2 to smithi17529310-29 2024-09-15T01:32:41.199 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 29 v 17 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-15T01:32:41.199 INFO:tasks.rados.rados.0.smithi175.stdout:71: finishing write tid 1 to smithi17529310-14 2024-09-15T01:32:41.199 INFO:tasks.rados.rados.0.smithi175.stdout:71: finishing write tid 2 to smithi17529310-14 2024-09-15T01:32:41.199 INFO:tasks.rados.rados.0.smithi175.stdout:71: finishing write tid 3 to smithi17529310-14 2024-09-15T01:32:41.199 INFO:tasks.rados.rados.0.smithi175.stdout:71: finishing write tid 4 to smithi17529310-14 2024-09-15T01:32:41.199 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 14 v 12 (ObjNum 56 snap 3 seq_num 56) dirty exists 2024-09-15T01:32:41.199 INFO:tasks.rados.rados.0.smithi175.stdout:71: left oid 14 (ObjNum 56 snap 3 seq_num 56) 2024-09-15T01:32:41.200 INFO:tasks.rados.rados.0.smithi175.stdout:72: finishing write tid 1 to smithi17529310-32 2024-09-15T01:32:41.200 INFO:tasks.rados.rados.0.smithi175.stdout:72: finishing write tid 2 to smithi17529310-32 2024-09-15T01:32:41.200 INFO:tasks.rados.rados.0.smithi175.stdout:72: finishing write tid 3 to smithi17529310-32 2024-09-15T01:32:41.200 INFO:tasks.rados.rados.0.smithi175.stdout:72: finishing write tid 4 to smithi17529310-32 2024-09-15T01:32:41.200 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 32 v 12 (ObjNum 57 snap 3 seq_num 57) dirty exists 2024-09-15T01:32:41.200 INFO:tasks.rados.rados.0.smithi175.stdout:72: left oid 32 (ObjNum 57 snap 3 seq_num 57) 2024-09-15T01:32:41.200 INFO:tasks.rados.rados.0.smithi175.stdout:70: done (4 left) 2024-09-15T01:32:41.200 INFO:tasks.rados.rados.0.smithi175.stdout:71: done (3 left) 2024-09-15T01:32:41.200 INFO:tasks.rados.rados.0.smithi175.stdout:72: done (2 left) 2024-09-15T01:32:41.200 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 40 current snap is 3 2024-09-15T01:32:41.200 INFO:tasks.rados.rados.0.smithi175.stdout:75: seq_num 58 ranges {671744=606208,1277952=376832} 2024-09-15T01:32:41.206 INFO:tasks.rados.rados.0.smithi175.stdout:75: writing smithi17529310-40 from 671744 to 1277952 tid 1 2024-09-15T01:32:41.209 INFO:tasks.rados.rados.0.smithi175.stdout:75: writing smithi17529310-40 from 1277952 to 1654784 tid 2 2024-09-15T01:32:41.209 INFO:tasks.rados.rados.0.smithi175.stdout:76: copy_from oid 43 from oid 9 current snap is 3 2024-09-15T01:32:41.209 INFO:tasks.rados.rados.0.smithi175.stdout:77: read oid 44 snap 0 2024-09-15T01:32:41.209 INFO:tasks.rados.rados.0.smithi175.stdout:77: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T01:32:41.209 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 3 current snap is 3 2024-09-15T01:32:41.209 INFO:tasks.rados.rados.0.smithi175.stdout:78: seq_num 59 ranges {49152=622592,671744=376832} 2024-09-15T01:32:41.214 INFO:tasks.rados.rados.0.smithi175.stdout:78: writing smithi17529310-3 from 49152 to 671744 tid 1 2024-09-15T01:32:41.217 INFO:tasks.rados.rados.0.smithi175.stdout:78: writing smithi17529310-3 from 671744 to 1048576 tid 2 2024-09-15T01:32:41.217 INFO:tasks.rados.rados.0.smithi175.stdout:79: delete oid 27 current snap is 3 2024-09-15T01:32:41.217 INFO:tasks.rados.rados.0.smithi175.stdout:69: finishing rollback tid 0 to smithi17529310-30 2024-09-15T01:32:41.217 INFO:tasks.rados.rados.0.smithi175.stdout:75: finishing write tid 1 to smithi17529310-40 2024-09-15T01:32:41.217 INFO:tasks.rados.rados.0.smithi175.stdout:69: finishing rollback tid 1 to smithi17529310-30 2024-09-15T01:32:41.217 INFO:tasks.rados.rados.0.smithi175.stdout:69: finishing rollback tid 2 to smithi17529310-30 2024-09-15T01:32:41.217 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 30 v 8 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-15T01:32:41.217 INFO:tasks.rados.rados.0.smithi175.stdout:75: finishing write tid 2 to smithi17529310-40 2024-09-15T01:32:41.218 INFO:tasks.rados.rados.0.smithi175.stdout:75: finishing write tid 3 to smithi17529310-40 2024-09-15T01:32:41.218 INFO:tasks.rados.rados.0.smithi175.stdout:69: done (6 left) 2024-09-15T01:32:41.218 INFO:tasks.rados.rados.0.smithi175.stdout:79: done (5 left) 2024-09-15T01:32:41.218 INFO:tasks.rados.rados.0.smithi175.stdout:80: setattr oid 2 current snap is 3 2024-09-15T01:32:41.219 INFO:tasks.rados.rados.0.smithi175.stdout:77: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T01:32:41.239 INFO:tasks.rados.rados.0.smithi175.stdout:75: finishing write tid 4 to smithi17529310-40 2024-09-15T01:32:41.239 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 40 v 10 (ObjNum 58 snap 3 seq_num 58) dirty exists 2024-09-15T01:32:41.239 INFO:tasks.rados.rados.0.smithi175.stdout:75: left oid 40 (ObjNum 58 snap 3 seq_num 58) 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:76: finishing copy_from racing read to smithi17529310-43 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:78: finishing write tid 1 to smithi17529310-3 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:76: finishing copy_from to smithi17529310-43 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 43 v 4 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:78: finishing write tid 2 to smithi17529310-3 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:78: finishing write tid 3 to smithi17529310-3 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:78: finishing write tid 4 to smithi17529310-3 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 3 v 13 (ObjNum 59 snap 3 seq_num 59) dirty exists 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:78: left oid 3 (ObjNum 59 snap 3 seq_num 59) 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:75: done (5 left) 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:76: done (4 left) 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:77: done (3 left) 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:78: done (2 left) 2024-09-15T01:32:41.240 INFO:tasks.rados.rados.0.smithi175.stdout:81: delete oid 13 current snap is 3 2024-09-15T01:32:41.243 INFO:tasks.rados.rados.0.smithi175.stdout:81: done (2 left) 2024-09-15T01:32:41.243 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 21 current snap is 3 2024-09-15T01:32:41.243 INFO:tasks.rados.rados.0.smithi175.stdout:82: seq_num 62 ranges {344064=671744,1015808=376832} 2024-09-15T01:32:41.249 INFO:tasks.rados.rados.0.smithi175.stdout:82: writing smithi17529310-21 from 344064 to 1015808 tid 1 2024-09-15T01:32:41.251 INFO:tasks.rados.rados.0.smithi175.stdout:82: writing smithi17529310-21 from 1015808 to 1392640 tid 2 2024-09-15T01:32:41.251 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 32 current snap is 3 2024-09-15T01:32:41.251 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 2 v 19 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-15T01:32:41.252 INFO:tasks.rados.rados.0.smithi175.stdout:83: seq_num 63 ranges {1490944=688128,2179072=376832} 2024-09-15T01:32:41.263 INFO:tasks.rados.rados.0.smithi175.stdout:83: writing smithi17529310-32 from 1490944 to 2179072 tid 1 2024-09-15T01:32:41.266 INFO:tasks.rados.rados.0.smithi175.stdout:83: writing smithi17529310-32 from 2179072 to 2555904 tid 2 2024-09-15T01:32:41.266 INFO:tasks.rados.rados.0.smithi175.stdout:80: done (3 left) 2024-09-15T01:32:41.266 INFO:tasks.rados.rados.0.smithi175.stdout:84: read oid 9 snap -1 2024-09-15T01:32:41.266 INFO:tasks.rados.rados.0.smithi175.stdout:84: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-15T01:32:41.266 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 48 current snap is 3 2024-09-15T01:32:41.266 INFO:tasks.rados.rados.0.smithi175.stdout:82: finishing write tid 1 to smithi17529310-21 2024-09-15T01:32:41.266 INFO:tasks.rados.rados.0.smithi175.stdout:82: finishing write tid 2 to smithi17529310-21 2024-09-15T01:32:41.266 INFO:tasks.rados.rados.0.smithi175.stdout:82: finishing write tid 3 to smithi17529310-21 2024-09-15T01:32:41.266 INFO:tasks.rados.rados.0.smithi175.stdout:82: finishing write tid 4 to smithi17529310-21 2024-09-15T01:32:41.266 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 21 v 22 (ObjNum 62 snap 3 seq_num 62) dirty exists 2024-09-15T01:32:41.266 INFO:tasks.rados.rados.0.smithi175.stdout:82: left oid 21 (ObjNum 62 snap 3 seq_num 62) 2024-09-15T01:32:41.267 INFO:tasks.rados.rados.0.smithi175.stdout:85: seq_num 64 ranges {1687552=704512,2392064=376832} 2024-09-15T01:32:41.278 INFO:tasks.rados.rados.0.smithi175.stdout:85: writing smithi17529310-48 from 1687552 to 2392064 tid 1 2024-09-15T01:32:41.281 INFO:tasks.rados.rados.0.smithi175.stdout:85: writing smithi17529310-48 from 2392064 to 2768896 tid 2 2024-09-15T01:32:41.281 INFO:tasks.rados.rados.0.smithi175.stdout:82: done (4 left) 2024-09-15T01:32:41.281 INFO:tasks.rados.rados.0.smithi175.stdout:86: copy_from oid 38 from oid 16 current snap is 3 2024-09-15T01:32:41.281 INFO:tasks.rados.rados.0.smithi175.stdout:83: finishing write tid 1 to smithi17529310-32 2024-09-15T01:32:41.281 INFO:tasks.rados.rados.0.smithi175.stdout:83: finishing write tid 2 to smithi17529310-32 2024-09-15T01:32:41.282 INFO:tasks.rados.rados.0.smithi175.stdout:83: finishing write tid 3 to smithi17529310-32 2024-09-15T01:32:41.282 INFO:tasks.rados.rados.0.smithi175.stdout:84: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-15T01:32:41.285 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:41.283+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:41.286 INFO:tasks.rados.rados.0.smithi175.stdout:83: finishing write tid 4 to smithi17529310-32 2024-09-15T01:32:41.286 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 32 v 17 (ObjNum 63 snap 3 seq_num 63) dirty exists 2024-09-15T01:32:41.286 INFO:tasks.rados.rados.0.smithi175.stdout:83: left oid 32 (ObjNum 63 snap 3 seq_num 63) 2024-09-15T01:32:41.286 INFO:tasks.rados.rados.0.smithi175.stdout:85: finishing write tid 1 to smithi17529310-48 2024-09-15T01:32:41.286 INFO:tasks.rados.rados.0.smithi175.stdout:85: finishing write tid 2 to smithi17529310-48 2024-09-15T01:32:41.286 INFO:tasks.rados.rados.0.smithi175.stdout:83: done (4 left) 2024-09-15T01:32:41.286 INFO:tasks.rados.rados.0.smithi175.stdout:84: done (3 left) 2024-09-15T01:32:41.286 INFO:tasks.rados.rados.0.smithi175.stdout:87: delete oid 47 current snap is 3 2024-09-15T01:32:41.287 INFO:tasks.rados.rados.0.smithi175.stdout:87: done (3 left) 2024-09-15T01:32:41.287 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 50 current snap is 3 2024-09-15T01:32:41.287 INFO:tasks.rados.rados.0.smithi175.stdout:88: seq_num 66 ranges {835584=737280,1572864=376832} 2024-09-15T01:32:41.296 INFO:tasks.rados.rados.0.smithi175.stdout:88: writing smithi17529310-50 from 835584 to 1572864 tid 1 2024-09-15T01:32:41.298 INFO:tasks.rados.rados.0.smithi175.stdout:88: writing smithi17529310-50 from 1572864 to 1949696 tid 2 2024-09-15T01:32:41.298 INFO:tasks.rados.rados.0.smithi175.stdout:89: delete oid 1 current snap is 3 2024-09-15T01:32:41.299 INFO:tasks.rados.rados.0.smithi175.stdout:86: finishing copy_from racing read to smithi17529310-38 2024-09-15T01:32:41.299 INFO:tasks.rados.rados.0.smithi175.stdout:85: finishing write tid 3 to smithi17529310-48 2024-09-15T01:32:41.299 INFO:tasks.rados.rados.0.smithi175.stdout:86: finishing copy_from to smithi17529310-38 2024-09-15T01:32:41.299 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 38 v 4 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-15T01:32:41.299 INFO:tasks.rados.rados.0.smithi175.stdout:85: finishing write tid 4 to smithi17529310-48 2024-09-15T01:32:41.299 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 48 v 22 (ObjNum 64 snap 3 seq_num 64) dirty exists 2024-09-15T01:32:41.299 INFO:tasks.rados.rados.0.smithi175.stdout:85: left oid 48 (ObjNum 64 snap 3 seq_num 64) 2024-09-15T01:32:41.299 INFO:tasks.rados.rados.0.smithi175.stdout:85: done (4 left) 2024-09-15T01:32:41.299 INFO:tasks.rados.rados.0.smithi175.stdout:86: done (3 left) 2024-09-15T01:32:41.299 INFO:tasks.rados.rados.0.smithi175.stdout:89: done (2 left) 2024-09-15T01:32:41.300 INFO:tasks.rados.rados.0.smithi175.stdout:90: snap_remove snap 1 2024-09-15T01:32:41.305 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:41.305 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:41.305 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:32:41.305 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:32:41.305 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:41.314 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-15T01:32:41.385 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:41.384+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:41.391 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:32:41.413 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:41.413 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:32:41.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:19 3.0s0 3:0dcf4e6a:::smithi17529310-7:head [append 0~114688,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.180068+0000", 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.230764711, 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.007205805, 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 19 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.414 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180068+0000", 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180068+0000", 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180070+0000", 2024-09-15T01:32:41.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8199999999999999e-06 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180286+0000", 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021592399999999999 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180300+0000", 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4409e-05 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180307+0000", 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1729999999999998e-06 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180497+0000", 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000189648 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180697+0000", 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019966100000000001 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181516+0000", 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00081953600000000005 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181666+0000", 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015011899999999999 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181715+0000", 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8612999999999997e-05 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181993+0000", 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00027802000000000002 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182560+0000", 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00056686200000000001 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187206+0000", 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046456980000000002 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187274+0000", 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8322000000000001e-05 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:20 3.0s0 3:0dcf4e6a:::smithi17529310-7:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.180395+0000", 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.2304379140000004, 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0071042090000000002, 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:41.420 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 20 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180395+0000", 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180395+0000", 2024-09-15T01:32:41.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180396+0000", 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.78e-07 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180401+0000", 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6210000000000003e-06 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180409+0000", 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5830000000000007e-06 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180414+0000", 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.853e-06 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181813+0000", 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0013985759999999999 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181895+0000", 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2309000000000001e-05 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182258+0000", 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00036276599999999999 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182357+0000", 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9593999999999995e-05 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182376+0000", 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.836e-05 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182401+0000", 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5885e-05 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182568+0000", 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016627800000000001 2024-09-15T01:32:41.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187416+0000", 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0048480529999999997 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187499+0000", 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3353e-05 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:21 3.0s0 3:0dcf4e6a:::smithi17529310-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:41.426 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.180423+0000", 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.2304096539999998, 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010009717, 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 21 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180423+0000", 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180423+0000", 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180424+0000", 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.99e-07 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180437+0000", 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3621999999999999e-05 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180446+0000", 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8070000000000006e-06 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180456+0000", 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0049e-05 2024-09-15T01:32:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182418+0000", 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0019619529999999998 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182510+0000", 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1484999999999996e-05 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187777+0000", 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0052676370000000004 2024-09-15T01:32:41.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187957+0000", 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180077 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190433+0000", 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0024754880000000001 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.431 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:22 3.9s0 3:96e0e10a:::smithi17529310-8:head [append 0~131072,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.180461+0000", 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.230371538, 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0070048200000000001, 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 22 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180461+0000", 2024-09-15T01:32:41.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180461+0000", 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180462+0000", 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0900000000000001e-07 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180589+0000", 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012706499999999999 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180601+0000", 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2323e-05 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180607+0000", 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3399999999999997e-06 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180652+0000", 2024-09-15T01:32:41.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5531999999999997e-05 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180812+0000", 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000160089 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181574+0000", 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076220200000000002 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181699+0000", 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012454299999999999 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181723+0000", 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3566000000000001e-05 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181994+0000", 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00027119699999999999 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182438+0000", 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044409499999999998 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187410+0000", 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049721970000000002 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.187466+0000", 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6062e-05 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.438 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:23 3.9s0 3:96e0e10a:::smithi17529310-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.180750+0000", 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.2300824849999996, 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.007730799, 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 23 2024-09-15T01:32:41.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180750+0000", 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180750+0000", 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180751+0000", 2024-09-15T01:32:41.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7499999999999999e-07 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180756+0000", 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7600000000000002e-06 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180766+0000", 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0158e-05 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180772+0000", 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8749999999999997e-06 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181768+0000", 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00099620700000000004 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181857+0000", 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8701000000000003e-05 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182232+0000", 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037499199999999999 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182312+0000", 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0142999999999995e-05 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182328+0000", 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.626e-05 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182370+0000", 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1678999999999999e-05 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182448+0000", 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7904999999999994e-05 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.188408+0000", 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059603809999999998 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.188481+0000", 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2962999999999999e-05 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:24 3.9s0 3:96e0e10a:::smithi17529310-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.180780+0000", 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.2300527629999998, 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010806214999999999, 2024-09-15T01:32:41.445 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 24 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180780+0000", 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180780+0000", 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180781+0000", 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8699999999999995e-07 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180783+0000", 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5689999999999999e-06 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180798+0000", 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4724e-05 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180803+0000", 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2499999999999997e-06 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182470+0000", 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016663800000000001 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182560+0000", 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0439000000000006e-05 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.188525+0000", 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059653579999999996 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.188602+0000", 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6297000000000005e-05 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.191586+0000", 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0029847110000000001 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:25 3.9s0 3:91afec5e:::smithi17529310-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.180810+0000", 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.230022999, 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082006059999999992, 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:41.450 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 25 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180810+0000", 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180810+0000", 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.180810+0000", 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9299999999999998e-07 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181000+0000", 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018970599999999999 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181009+0000", 2024-09-15T01:32:41.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8780000000000004e-06 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181015+0000", 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8069999999999996e-06 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182584+0000", 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015697109999999999 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.182718+0000", 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013340699999999999 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183339+0000", 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062063400000000001 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183472+0000", 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013308600000000001 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183474+0000", 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4569999999999999e-06 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183536+0000", 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2401000000000001e-05 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184334+0000", 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00079743700000000002 2024-09-15T01:32:41.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.188949+0000", 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046148389999999999 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189010+0000", 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1749999999999997e-05 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:26 3.9s0 3:91afec5e:::smithi17529310-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.181020+0000", 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.2298124130000003, 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0081861610000000008, 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 26 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181020+0000", 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181020+0000", 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181021+0000", 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:32:41.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181024+0000", 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1640000000000001e-06 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181031+0000", 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1940000000000003e-06 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181035+0000", 2024-09-15T01:32:41.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1370000000000004e-06 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183555+0000", 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.002520235 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183638+0000", 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2961999999999996e-05 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.183993+0000", 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035526399999999999 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184061+0000", 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7547999999999998e-05 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184088+0000", 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7334e-05 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184130+0000", 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1576999999999999e-05 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184339+0000", 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020921700000000001 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189122+0000", 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0047831899999999997 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189207+0000", 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4594999999999999e-05 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:27 3.9s0 3:91afec5e:::smithi17529310-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.181039+0000", 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.2297941730000002, 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010833323000000001, 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:32:41.463 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 27 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181039+0000", 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181039+0000", 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181039+0000", 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.84e-07 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181041+0000", 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.781e-06 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181047+0000", 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9680000000000003e-06 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.181051+0000", 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8850000000000001e-06 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184117+0000", 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030660100000000001 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184197+0000", 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0199999999999998e-05 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189254+0000", 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050566220000000002 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189303+0000", 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9329000000000002e-05 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.191872+0000", 2024-09-15T01:32:41.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025691440000000002 2024-09-15T01:32:41.468 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.468 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.468 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.469 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:40 3.9s0 3:98b2f2fd:::smithi17529310-14:head [append 0~229376,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:41.469 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.184139+0000", 2024-09-15T01:32:41.469 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.2266942839999997, 2024-09-15T01:32:41.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0053846320000000003, 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 40 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184139+0000", 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184139+0000", 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184140+0000", 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3379999999999999e-06 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184399+0000", 2024-09-15T01:32:41.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000258607 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184412+0000", 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3575e-05 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184418+0000", 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6980000000000003e-06 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184539+0000", 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000121308 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184666+0000", 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012712699999999999 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.185330+0000", 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066421700000000004 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.185438+0000", 2024-09-15T01:32:41.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107284 2024-09-15T01:32:41.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.185455+0000", 2024-09-15T01:32:41.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7419e-05 2024-09-15T01:32:41.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.185522+0000", 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6686999999999994e-05 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.186106+0000", 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058390299999999998 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189489+0000", 2024-09-15T01:32:41.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0033835620000000001 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.189523+0000", 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3906999999999999e-05 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:42 3.9s0 3:98b2f2fd:::smithi17529310-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.184451+0000", 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.2263813849999998, 2024-09-15T01:32:41.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.007581529, 2024-09-15T01:32:41.477 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.477 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:32:41.477 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.477 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.477 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.477 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 42 2024-09-15T01:32:41.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.477 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184451+0000", 2024-09-15T01:32:41.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184451+0000", 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184452+0000", 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7700000000000005e-07 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184454+0000", 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5210000000000001e-06 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184462+0000", 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8590000000000005e-06 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.184467+0000", 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8899999999999998e-06 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.186843+0000", 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0023760449999999998 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.186899+0000", 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6063000000000002e-05 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190081+0000", 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0031817529999999998 2024-09-15T01:32:41.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.190142+0000", 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0823999999999999e-05 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.192033+0000", 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001891097 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:74 3.0s0 3:0187269c:::smithi17529310-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.2002160159999997, 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.071208814999999995, 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 74 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1200000000000003e-07 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210620+0000", 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.278e-06 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210623+0000", 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5779999999999999e-06 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210626+0000", 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.295e-06 2024-09-15T01:32:41.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211745+0000", 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011191770000000001 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211880+0000", 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013497499999999999 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212353+0000", 2024-09-15T01:32:41.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000473616 2024-09-15T01:32:41.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212441+0000", 2024-09-15T01:32:41.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7268999999999994e-05 2024-09-15T01:32:41.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212454+0000", 2024-09-15T01:32:41.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3529999999999999e-05 2024-09-15T01:32:41.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212483+0000", 2024-09-15T01:32:41.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9224e-05 2024-09-15T01:32:41.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212755+0000", 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00027191599999999998 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.281770+0000", 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.069014438999999997 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.281826+0000", 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6005999999999999e-05 2024-09-15T01:32:41.493 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:75 3.0s0 3:0187269c:::smithi17529310-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.2001837760000003, 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.072451382999999994, 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.494 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 75 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4700000000000002e-07 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210652+0000", 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9160000000000001e-06 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210657+0000", 2024-09-15T01:32:41.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0529999999999997e-06 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210660+0000", 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4339999999999999e-06 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212500+0000", 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001839732 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212584+0000", 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4759999999999995e-05 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.282036+0000", 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.069451322999999995 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.282131+0000", 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.577e-05 2024-09-15T01:32:41.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.283100+0000", 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00096904800000000002 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpRead(3.ds2 34/31 ECSubRead(tid=14, to_read={3:b063f44c:::smithi17529310-36:head=0,4096,0}, subchunks={3:b063f44c:::smithi17529310-36:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.249402+0000", 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.1614313369999998, 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0097383939999999992, 2024-09-15T01:32:41.499 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.249402+0000", 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.249402+0000", 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.249402+0000", 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.249410+0000", 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7200000000000006e-06 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259023+0000", 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0096131719999999997 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259029+0000", 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0510000000000002e-06 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259073+0000", 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4422000000000003e-05 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259140+0000", 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6528999999999999e-05 2024-09-15T01:32:41.502 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds2 34/31 ECSubWrite(tid=16, reqid=client.4477.0:116, at_version=34'10, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.1512093270000001, 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.24562283800000001, 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9599999999999999e-07 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259634+0000", 2024-09-15T01:32:41.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0412e-05 2024-09-15T01:32:41.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.504838+0000", 2024-09-15T01:32:41.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.245203848 2024-09-15T01:32:41.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.504852+0000", 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3346999999999999e-05 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505000+0000", 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000148209 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505006+0000", 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6309999999999997e-06 2024-09-15T01:32:41.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505068+0000", 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1962000000000001e-05 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505074+0000", 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0579999999999997e-06 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505103+0000", 2024-09-15T01:32:41.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9828e-05 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505116+0000", 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2261e-05 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505246+0000", 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000130786 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.508 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as4 39/38 ECSubWrite(tid=14, reqid=client.4477.0:213, at_version=39'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:41.207986+0000", 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.202846634, 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0066366760000000002, 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.207986+0000", 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.207986+0000", 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.207987+0000", 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4900000000000005e-07 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208447+0000", 2024-09-15T01:32:41.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045987799999999999 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208471+0000", 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4516999999999999e-05 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208476+0000", 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6929999999999998e-06 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208519+0000", 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2981000000000001e-05 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208531+0000", 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1626e-05 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208735+0000", 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020436900000000001 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208745+0000", 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0169999999999999e-05 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.210057+0000", 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001311687 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.210075+0000", 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8176e-05 2024-09-15T01:32:41.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.214623+0000", 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045477299999999998 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:220 3.9s0 3:901afd6f:::smithi17529310-44:2 [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:41.208710+0000", 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.20212308800000001, 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0074051029999999997, 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.514 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 220 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208710+0000", 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208710+0000", 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208710+0000", 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208713+0000", 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5189999999999999e-06 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208718+0000", 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6690000000000003e-06 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208722+0000", 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4249999999999998e-06 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.209065+0000", 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034289699999999999 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.209122+0000", 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7599999999999997e-05 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.216115+0000", 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0069926069999999996 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:221 3.9s0 3:901afd6f:::smithi17529310-44:2 [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:41.518 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:41.208734+0000", 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.20209885799999999, 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0079374779999999995, 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 221 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208734+0000", 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208734+0000", 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208734+0000", 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6399999999999998e-07 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208737+0000", 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3140000000000002e-06 2024-09-15T01:32:41.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208742+0000", 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6790000000000002e-06 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208746+0000", 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5200000000000002e-06 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.209863+0000", 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001117259 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.209961+0000", 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7555999999999999e-05 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.216671+0000", 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0067107859999999998 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpReadReply(3.9s0 39/31 ECSubReadReply(tid=34, attrs_read=0))", 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:41.209887+0000", 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.200946133, 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0062618459999999997, 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.209887+0000", 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.209887+0000", 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.209887+0000", 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.03e-07 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.210574+0000", 2024-09-15T01:32:41.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00068722800000000001 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.215548+0000", 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049740339999999996 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.215556+0000", 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6529999999999994e-06 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.215595+0000", 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8915000000000003e-05 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.216149+0000", 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000553613 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as4 39/38 ECSubWrite(tid=15, reqid=client.4477.0:214, at_version=39'9, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:41.526 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:41.210578+0000", 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.20025490800000001, 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0054762689999999998, 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.210578+0000", 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.210578+0000", 2024-09-15T01:32:41.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.210578+0000", 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7e-07 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.210624+0000", 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5745999999999999e-05 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:41.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.215571+0000", 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004946654 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.215574+0000", 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.05e-06 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.215620+0000", 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5803999999999997e-05 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.215629+0000", 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9239999999999996e-06 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.215740+0000", 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011090300000000001 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.215748+0000", 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0749999999999998e-06 2024-09-15T01:32:41.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.215830+0000", 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2490999999999993e-05 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.216027+0000", 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000197037 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:41.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.216054+0000", 2024-09-15T01:32:41.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7115e-05 2024-09-15T01:32:41.532 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.532 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.532 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.532 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:41.532 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:41.532 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:41.532 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:32:41.534 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:32:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:32:41.534 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:41.532+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:41.544 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:32:41.544 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:32:41.564 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:32:41.565 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:32:41.589 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:32:41.589 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:32:41.607 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:32:41.608 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:32:41.627 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:32:41.627 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:32:41.635 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:41.633+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:41.635 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:41.635 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:41.643 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:32:41.647 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:32:41.648 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:32:41.671 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:32:41.671 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:32:41.689 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:32:41.689 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:32:41.708 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:32:41.708 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:32:41.727 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:32:41.727 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:32:41.736 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:41.734+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:41.740 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:41.740 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:41.746 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:32:41.746 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:32:41.749 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:32:41.764 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:32:41.764 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:32:41.784 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:32:41.784 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:32:41.803 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:32:41.803 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:32:41.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:32:41.802+0000 7f7d2ad9d640 0 -- 172.21.15.42:0/4217692277 >> [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] conn(0x7f7d2c1981c0 msgr2=0x7f7d2c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:32:41.837 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:41.835+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:41.845 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:41.845 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:41.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:32:41.938 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:41.936+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:41.953 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:41.953 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:41.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:32:42.023 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:32:42.024 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:32:42.038 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:42.037+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:42.059 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:42.059 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:42.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:32:42.139 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:42.138+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:42.199 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15] out_osds: [9] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-15T01:32:42.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T01:32:42.205 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:42.205 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:42.213 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:32:42.217 INFO:tasks.rados.rados.0.smithi175.stdout:90: done (2 left) 2024-09-15T01:32:42.217 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 29 current snap is 3 2024-09-15T01:32:42.217 INFO:tasks.rados.rados.0.smithi175.stdout:88: finishing write tid 1 to smithi17529310-50 2024-09-15T01:32:42.217 INFO:tasks.rados.rados.0.smithi175.stdout:91: seq_num 68 ranges {475136=770048,1245184=376832} 2024-09-15T01:32:42.219 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:32:42.225 INFO:tasks.rados.rados.0.smithi175.stdout:91: writing smithi17529310-29 from 475136 to 1245184 tid 1 2024-09-15T01:32:42.228 INFO:tasks.rados.rados.0.smithi175.stdout:91: writing smithi17529310-29 from 1245184 to 1622016 tid 2 2024-09-15T01:32:42.228 INFO:tasks.rados.rados.0.smithi175.stdout:92: rollback oid 25 current snap is 3 2024-09-15T01:32:42.228 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 25 to 0 2024-09-15T01:32:42.228 INFO:tasks.rados.rados.0.smithi175.stdout:88: finishing write tid 2 to smithi17529310-50 2024-09-15T01:32:42.228 INFO:tasks.rados.rados.0.smithi175.stdout:74: finishing copy_from to smithi17529310-6 2024-09-15T01:32:42.228 INFO:tasks.rados.rados.0.smithi175.stdout:74: got expected ENOENT (src dne) 2024-09-15T01:32:42.228 INFO:tasks.rados.rados.0.smithi175.stdout:74: finishing copy_from racing read to smithi17529310-6 2024-09-15T01:32:42.228 INFO:tasks.rados.rados.0.smithi175.stdout:88: finishing write tid 3 to smithi17529310-50 2024-09-15T01:32:42.228 INFO:tasks.rados.rados.0.smithi175.stdout:88: finishing write tid 4 to smithi17529310-50 2024-09-15T01:32:42.228 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 50 v 11 (ObjNum 66 snap 3 seq_num 66) dirty exists 2024-09-15T01:32:42.228 INFO:tasks.rados.rados.0.smithi175.stdout:88: left oid 50 (ObjNum 66 snap 3 seq_num 66) 2024-09-15T01:32:42.229 INFO:tasks.rados.rados.0.smithi175.stdout:74: done (3 left) 2024-09-15T01:32:42.229 INFO:tasks.rados.rados.0.smithi175.stdout:88: done (2 left) 2024-09-15T01:32:42.229 INFO:tasks.rados.rados.0.smithi175.stdout:93: delete oid 49 current snap is 3 2024-09-15T01:32:42.231 INFO:tasks.rados.rados.0.smithi175.stdout:92: finishing rollback tid 0 to smithi17529310-25 2024-09-15T01:32:42.232 INFO:tasks.rados.rados.0.smithi175.stdout:92: finishing rollback tid 1 to smithi17529310-25 2024-09-15T01:32:42.233 INFO:tasks.rados.rados.0.smithi175.stdout:92: finishing rollback tid 2 to smithi17529310-25 2024-09-15T01:32:42.233 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 25 v 14 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-15T01:32:42.235 INFO:tasks.rados.rados.0.smithi175.stdout:91: finishing write tid 1 to smithi17529310-29 2024-09-15T01:32:42.236 INFO:tasks.rados.rados.0.smithi175.stdout:91: finishing write tid 2 to smithi17529310-29 2024-09-15T01:32:42.236 INFO:tasks.rados.rados.0.smithi175.stdout:91: finishing write tid 3 to smithi17529310-29 2024-09-15T01:32:42.237 INFO:tasks.rados.rados.0.smithi175.stdout:92: done (2 left) 2024-09-15T01:32:42.237 INFO:tasks.rados.rados.0.smithi175.stdout:93: done (1 left) 2024-09-15T01:32:42.237 INFO:tasks.rados.rados.0.smithi175.stdout:94: read oid 7 snap -1 2024-09-15T01:32:42.237 INFO:tasks.rados.rados.0.smithi175.stdout:94: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-15T01:32:42.237 INFO:tasks.rados.rados.0.smithi175.stdout:95: snap_create 2024-09-15T01:32:42.238 INFO:tasks.rados.rados.0.smithi175.stdout:91: finishing write tid 4 to smithi17529310-29 2024-09-15T01:32:42.238 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 29 v 26 (ObjNum 68 snap 3 seq_num 68) dirty exists 2024-09-15T01:32:42.238 INFO:tasks.rados.rados.0.smithi175.stdout:91: left oid 29 (ObjNum 68 snap 3 seq_num 68) 2024-09-15T01:32:42.240 INFO:tasks.rados.rados.0.smithi175.stdout:94: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-15T01:32:42.240 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:42.238+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:42.315 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:42.315 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:42.323 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:32:42.365 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:42.363+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:42.441 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:42.440+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:42.461 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:42.469 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:32:42.542 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:42.540+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:42.566 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:42.567 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:42.575 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-15T01:32:42.642 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:42.641+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:42.672 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:42.672 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:42.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-15T01:32:42.743 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:42.742+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:42.817 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:42.817 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:42.826 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-15T01:32:42.844 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:42.842+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:42.922 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:42.922 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:42.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:32:42.977 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:42.975+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:43.038 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:43.039 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:43.046 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:43.044+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:43.047 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:32:43.147 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:43.146+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:43.188 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:43.189 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:43.197 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:32:43.220 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T01:32:43.221 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-15T01:32:43.222 INFO:tasks.rados.rados.0.smithi175.stdout:91: done (2 left) 2024-09-15T01:32:43.222 INFO:tasks.rados.rados.0.smithi175.stdout:94: done (1 left) 2024-09-15T01:32:43.222 INFO:tasks.rados.rados.0.smithi175.stdout:95: done (0 left) 2024-09-15T01:32:43.222 INFO:tasks.rados.rados.0.smithi175.stdout:96: read oid 6 snap 3 2024-09-15T01:32:43.222 INFO:tasks.rados.rados.0.smithi175.stdout:96: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-15T01:32:43.222 INFO:tasks.rados.rados.0.smithi175.stdout:97: read oid 11 snap -1 2024-09-15T01:32:43.222 INFO:tasks.rados.rados.0.smithi175.stdout:97: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T01:32:43.223 INFO:tasks.rados.rados.0.smithi175.stdout:98: read oid 41 snap 3 2024-09-15T01:32:43.223 INFO:tasks.rados.rados.0.smithi175.stdout:98: expect (ObjNum 51 snap 1 seq_num 51) 2024-09-15T01:32:43.223 INFO:tasks.rados.rados.0.smithi175.stdout:99: snap_create 2024-09-15T01:32:43.228 INFO:tasks.rados.rados.0.smithi175.stdout:97: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T01:32:43.233 INFO:tasks.rados.rados.0.smithi175.stdout:96: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-15T01:32:43.236 INFO:tasks.rados.rados.0.smithi175.stdout:98: expect (ObjNum 51 snap 1 seq_num 51) 2024-09-15T01:32:43.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T01:32:43.248 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:43.247+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:43.304 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:43.304 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:43.314 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:32:43.349 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:43.347+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:43.449 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:43.447+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:43.456 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:43.457 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:43.465 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:32:43.550 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:43.549+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:43.566 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:43.566 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:43.575 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:32:43.651 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:43.650+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:43.675 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:43.676 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:43.684 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:32:43.768 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:43.767+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:43.826 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:43.826 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:43.834 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:32:43.852 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:43.851+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:43.953 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:43.952+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:43.976 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:43.976 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:43.984 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:32:44.035 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:32:44.054 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:44.052+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:44.086 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:44.086 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:44.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-15T01:32:44.154 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:44.153+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:44.176 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:32:44.177 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:32:44.196 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:32:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:32:44.208 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:44.208 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:44.216 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:32:44.216 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:32:44.217 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-15T01:32:44.224 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T01:32:44.226 INFO:tasks.rados.rados.0.smithi175.stdout:96: done (3 left) 2024-09-15T01:32:44.226 INFO:tasks.rados.rados.0.smithi175.stdout:97: done (2 left) 2024-09-15T01:32:44.226 INFO:tasks.rados.rados.0.smithi175.stdout:98: done (1 left) 2024-09-15T01:32:44.226 INFO:tasks.rados.rados.0.smithi175.stdout:99: done (0 left) 2024-09-15T01:32:44.226 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 7 current snap is 5 2024-09-15T01:32:44.226 INFO:tasks.rados.rados.0.smithi175.stdout:100: seq_num 70 ranges {114688=802816,917504=376832} 2024-09-15T01:32:44.231 INFO:tasks.rados.rados.0.smithi175.stdout:100: writing smithi17529310-7 from 114688 to 917504 tid 1 2024-09-15T01:32:44.233 INFO:tasks.rados.rados.0.smithi175.stdout:100: writing smithi17529310-7 from 917504 to 1294336 tid 2 2024-09-15T01:32:44.233 INFO:tasks.rados.rados.0.smithi175.stdout:101: snap_remove snap 3 2024-09-15T01:32:44.237 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:32:44.237 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:32:44.238 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T01:32:44.255 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:44.254+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:44.256 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:32:44.256 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:32:44.278 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:32:44.278 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:32:44.311 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:32:44.311 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:32:44.343 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:32:44.343 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:32:44.348 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:44.348 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:44.356 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:44.355+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:44.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-15T01:32:44.367 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:32:44.367 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:32:44.399 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:32:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:32:44.432 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:32:44.432 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:32:44.457 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:44.455+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:44.466 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:32:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:32:44.468 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:44.469 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:44.478 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:32:44.501 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:32:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:32:44.527 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:32:44.527 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:32:44.548 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:32:44.548 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:32:44.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:32:44.546+0000 7f68fe0fa640 0 -- 172.21.15.42:0/1202118938 >> [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] conn(0x7f68f8195d00 msgr2=0x7f68f81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:32:44.558 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:44.556+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:44.589 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:32:44.589 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:44.598 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:32:44.659 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:44.657+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:44.695 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:32:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:32:44.695 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:44.703 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:32:44.759 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:44.757+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:44.768 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:32:44.769 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:32:44.801 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 34/31 ECSubWrite(tid=9, reqid=client.4477.0:124, at_version=34'5, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.271871+0000", 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.5271035289999997, 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034356310000000002, 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.271871+0000", 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.271871+0000", 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.271872+0000", 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4000000000000001e-07 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.271934+0000", 2024-09-15T01:32:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1617000000000003e-05 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.271967+0000", 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3259999999999997e-05 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.271974+0000", 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3989999999999998e-06 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272015+0000", 2024-09-15T01:32:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272024+0000", 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4360000000000008e-06 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272278+0000", 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025420400000000001 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272289+0000", 2024-09-15T01:32:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0550000000000001e-05 2024-09-15T01:32:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272409+0000", 2024-09-15T01:32:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119882 2024-09-15T01:32:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272430+0000", 2024-09-15T01:32:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1206e-05 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.275307+0000", 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028768919999999998 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 34/31 ECSubWrite(tid=10, reqid=client.4477.0:125, at_version=34'6, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.272523+0000", 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.5264516589999992, 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028933019999999999, 2024-09-15T01:32:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272523+0000", 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272523+0000", 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272524+0000", 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6100000000000001e-07 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272535+0000", 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1379000000000001e-05 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272564+0000", 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9170999999999999e-05 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272570+0000", 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9190000000000004e-06 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272617+0000", 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6839999999999999e-05 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272628+0000", 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0787e-05 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272711+0000", 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3067999999999998e-05 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272718+0000", 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7000000000000002e-06 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272751+0000", 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3912999999999999e-05 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272766+0000", 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4718e-05 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.275416+0000", 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026502460000000002 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 34/31 ECSubWrite(tid=11, reqid=client.4477.0:125, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:32:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.273456+0000", 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.5255191359999998, 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020113829999999998, 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273456+0000", 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273456+0000", 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273456+0000", 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3200000000000005e-07 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273463+0000", 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9070000000000001e-06 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273494+0000", 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0806000000000003e-05 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273500+0000", 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273538+0000", 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8569000000000003e-05 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273543+0000", 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1710000000000001e-06 2024-09-15T01:32:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273609+0000", 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5326000000000004e-05 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273617+0000", 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4619999999999996e-06 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273647+0000", 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9487000000000001e-05 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273658+0000", 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1212999999999999e-05 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.275467+0000", 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001809267 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs2 34/31 ECSubWrite(tid=24, reqid=client.4477.0:151, at_version=34'4, trim_to=0'0, roll_forward_to=34'1))", 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.309229+0000", 2024-09-15T01:32:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.4897459489999996, 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002541967, 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.309229+0000", 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.309229+0000", 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.309230+0000", 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8199999999999999e-07 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.309425+0000", 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019526 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.309447+0000", 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2402e-05 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.309455+0000", 2024-09-15T01:32:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4610000000000002e-06 2024-09-15T01:32:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.309495+0000", 2024-09-15T01:32:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9947000000000002e-05 2024-09-15T01:32:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.309501+0000", 2024-09-15T01:32:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4590000000000003e-06 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.309583+0000", 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1627999999999998e-05 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.309589+0000", 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.311557+0000", 2024-09-15T01:32:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019677660000000001 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.311584+0000", 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6764e-05 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.311771+0000", 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000187042 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs2 37/36 ECSubWrite(tid=30, reqid=client.4477.0:185, at_version=37'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:39.189139+0000", 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.6098356169999999, 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018096189999999999, 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.189139+0000", 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.189139+0000", 2024-09-15T01:32:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.189140+0000", 2024-09-15T01:32:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4300000000000003e-07 2024-09-15T01:32:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.189160+0000", 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0704e-05 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.189188+0000", 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7623999999999998e-05 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.189193+0000", 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.476e-06 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.189222+0000", 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9229000000000001e-05 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.189228+0000", 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6939999999999998e-06 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.189435+0000", 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020628199999999999 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.189443+0000", 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0450000000000001e-06 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.189483+0000", 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0737e-05 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.189497+0000", 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3223e-05 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.190949+0000", 2024-09-15T01:32:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014520620000000001 2024-09-15T01:32:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as2 37/36 ECSubWrite(tid=12, reqid=client.4477.0:192, at_version=37'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:39.201865+0000", 2024-09-15T01:32:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.5971099950000003, 2024-09-15T01:32:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019505620000000001, 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201865+0000", 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201865+0000", 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201866+0000", 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4700000000000001e-07 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.202304+0000", 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000438164 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.202330+0000", 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6097000000000001e-05 2024-09-15T01:32:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.202336+0000", 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0530000000000003e-06 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.202370+0000", 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4137999999999997e-05 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.202377+0000", 2024-09-15T01:32:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1289999999999998e-06 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.202616+0000", 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000238905 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.202624+0000", 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3170000000000002e-06 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.202711+0000", 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6223000000000004e-05 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.202722+0000", 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1412e-05 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.203815+0000", 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001093377 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.9s1 38/31 ECSubWrite(tid=26, reqid=client.4477.0:202, at_version=38'10, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.210032+0000", 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.5889427380000001, 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039483310000000002, 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.210032+0000", 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.210032+0000", 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.210032+0000", 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.210429+0000", 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039626700000000001 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.210452+0000", 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3249000000000001e-05 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.210458+0000", 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6509999999999998e-06 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.210511+0000", 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2942999999999998e-05 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.210518+0000", 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5989999999999996e-06 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.210776+0000", 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025807599999999999 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.210784+0000", 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9249999999999995e-06 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.213142+0000", 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023573790000000002 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.213159+0000", 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7725999999999999e-05 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.213980+0000", 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00082107600000000001 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.9s1 38/31 ECSubWrite(tid=27, reqid=client.4477.0:203, at_version=38'11, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.211541+0000", 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.5874337040000004, 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002622624, 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.211541+0000", 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.211541+0000", 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.211542+0000", 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9699999999999996e-07 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.212098+0000", 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055681199999999995 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.212118+0000", 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9684e-05 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.212123+0000", 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2429999999999996e-06 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.212157+0000", 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3986999999999998e-05 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.212164+0000", 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.144e-06 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.212249+0000", 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5408000000000006e-05 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.213146+0000", 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00089714199999999997 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.213214+0000", 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7438e-05 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.213231+0000", 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7402999999999999e-05 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.214164+0000", 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093276600000000002 2024-09-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 38/31 ECSubWrite(tid=15, reqid=client.4477.0:206, at_version=38'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.219279+0000", 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.5796959189999997, 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005050237, 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219279+0000", 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219279+0000", 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219279+0000", 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5499999999999999e-07 2024-09-15T01:32:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219372+0000", 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2876999999999999e-05 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219420+0000", 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7666000000000001e-05 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219427+0000", 2024-09-15T01:32:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-15T01:32:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219491+0000", 2024-09-15T01:32:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4499999999999996e-05 2024-09-15T01:32:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219500+0000", 2024-09-15T01:32:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.772e-06 2024-09-15T01:32:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219751+0000", 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025104499999999998 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219761+0000", 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8859999999999999e-06 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219848+0000", 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7251999999999997e-05 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219882+0000", 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3688e-05 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.224329+0000", 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044470400000000002 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 38/31 ECSubWrite(tid=16, reqid=client.4477.0:207, at_version=38'11, trim_to=0'0, roll_forward_to=36'8))", 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.220656+0000", 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.5783191749999999, 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040216460000000002, 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220656+0000", 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220656+0000", 2024-09-15T01:32:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220657+0000", 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6200000000000006e-07 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220714+0000", 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6975999999999998e-05 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220748+0000", 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4489999999999997e-05 2024-09-15T01:32:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220755+0000", 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0960000000000003e-06 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220801+0000", 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6171999999999998e-05 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220810+0000", 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8480000000000007e-06 2024-09-15T01:32:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220900+0000", 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9993000000000004e-05 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220907+0000", 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0509999999999999e-06 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220978+0000", 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0436000000000001e-05 2024-09-15T01:32:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220995+0000", 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7785e-05 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.224677+0000", 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036818369999999999 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 38/31 ECSubWrite(tid=17, reqid=client.4477.0:208, at_version=38'12, trim_to=0'0, roll_forward_to=36'8))", 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.221300+0000", 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.5776748080000003, 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036497719999999999, 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221300+0000", 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221300+0000", 2024-09-15T01:32:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221301+0000", 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1299999999999999e-07 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221320+0000", 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9034000000000001e-05 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221364+0000", 2024-09-15T01:32:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4180000000000001e-05 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221371+0000", 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6789999999999999e-06 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221417+0000", 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6622999999999997e-05 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221425+0000", 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2719999999999997e-06 2024-09-15T01:32:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221506+0000", 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0117000000000006e-05 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221531+0000", 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5636e-05 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221569+0000", 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7814000000000001e-05 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221579+0000", 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7669999999999998e-06 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.224950+0000", 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033709370000000001 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 39/31 ECSubWrite(tid=20, reqid=osd.5.0:20, at_version=39'14, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T01:32:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.195627+0000", 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.603347683, 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019440169999999999, 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.195627+0000", 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.195627+0000", 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.195627+0000", 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.15e-07 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.195656+0000", 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8365e-05 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.195723+0000", 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7522999999999997e-05 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.195743+0000", 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9755999999999999e-05 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.195813+0000", 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9999999999999994e-05 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.195823+0000", 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5550000000000008e-06 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.196055+0000", 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000232143 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.196066+0000", 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0878e-05 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.196101+0000", 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5110000000000001e-05 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.196134+0000", 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3337000000000003e-05 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.197571+0000", 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014370349999999999 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as2 39/38 ECSubWrite(tid=14, reqid=client.4477.0:213, at_version=39'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.207646+0000", 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.5913284729999999, 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031903349999999999, 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.207646+0000", 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.207646+0000", 2024-09-15T01:32:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.207647+0000", 2024-09-15T01:32:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-15T01:32:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.208004+0000", 2024-09-15T01:32:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035696799999999999 2024-09-15T01:32:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.208035+0000", 2024-09-15T01:32:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0793999999999997e-05 2024-09-15T01:32:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.208041+0000", 2024-09-15T01:32:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1850000000000001e-06 2024-09-15T01:32:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.208085+0000", 2024-09-15T01:32:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3800000000000001e-05 2024-09-15T01:32:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.208094+0000", 2024-09-15T01:32:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-15T01:32:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.208382+0000", 2024-09-15T01:32:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028778799999999998 2024-09-15T01:32:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.208390+0000", 2024-09-15T01:32:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9619999999999993e-06 2024-09-15T01:32:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209011+0000", 2024-09-15T01:32:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062073200000000003 2024-09-15T01:32:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209038+0000", 2024-09-15T01:32:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6976999999999999e-05 2024-09-15T01:32:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.210837+0000", 2024-09-15T01:32:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001799113 2024-09-15T01:32:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s2 39/38 ECSubWrite(tid=12, reqid=client.4477.0:197, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.207725+0000", 2024-09-15T01:32:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.5912493909999998, 2024-09-15T01:32:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031571849999999999, 2024-09-15T01:32:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.207725+0000", 2024-09-15T01:32:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.207725+0000", 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.207726+0000", 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0299999999999999e-07 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.207747+0000", 2024-09-15T01:32:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1047e-05 2024-09-15T01:32:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.207875+0000", 2024-09-15T01:32:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000127954 2024-09-15T01:32:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.207882+0000", 2024-09-15T01:32:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4699999999999996e-06 2024-09-15T01:32:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.207929+0000", 2024-09-15T01:32:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6817999999999998e-05 2024-09-15T01:32:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.207939+0000", 2024-09-15T01:32:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0147e-05 2024-09-15T01:32:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.208023+0000", 2024-09-15T01:32:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3327999999999999e-05 2024-09-15T01:32:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.208165+0000", 2024-09-15T01:32:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000142767 2024-09-15T01:32:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.208203+0000", 2024-09-15T01:32:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7827999999999998e-05 2024-09-15T01:32:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.208226+0000", 2024-09-15T01:32:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3088999999999999e-05 2024-09-15T01:32:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.210883+0000", 2024-09-15T01:32:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026562339999999999 2024-09-15T01:32:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s2 39/38 ECSubWrite(tid=13, reqid=client.4477.0:198, at_version=39'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.209109+0000", 2024-09-15T01:32:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.589865627, 2024-09-15T01:32:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018737020000000001, 2024-09-15T01:32:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209109+0000", 2024-09-15T01:32:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209109+0000", 2024-09-15T01:32:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209110+0000", 2024-09-15T01:32:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.54e-07 2024-09-15T01:32:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209129+0000", 2024-09-15T01:32:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9128e-05 2024-09-15T01:32:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209159+0000", 2024-09-15T01:32:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9811e-05 2024-09-15T01:32:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209167+0000", 2024-09-15T01:32:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5959999999999998e-06 2024-09-15T01:32:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209227+0000", 2024-09-15T01:32:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0068999999999997e-05 2024-09-15T01:32:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209237+0000", 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0502999999999999e-05 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209347+0000", 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109518 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209768+0000", 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00042156000000000002 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209823+0000", 2024-09-15T01:32:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5009999999999997e-05 2024-09-15T01:32:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.209838+0000", 2024-09-15T01:32:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5245999999999999e-05 2024-09-15T01:32:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.210983+0000", 2024-09-15T01:32:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011444070000000001 2024-09-15T01:32:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 39/31 ECSubWrite(tid=21, reqid=client.4477.0:217, at_version=39'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-15T01:32:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.219457+0000", 2024-09-15T01:32:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.5795182370000003, 2024-09-15T01:32:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003381383, 2024-09-15T01:32:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219457+0000", 2024-09-15T01:32:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219457+0000", 2024-09-15T01:32:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219457+0000", 2024-09-15T01:32:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.68e-07 2024-09-15T01:32:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219676+0000", 2024-09-15T01:32:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021907200000000001 2024-09-15T01:32:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219725+0000", 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9184999999999997e-05 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219733+0000", 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7849999999999992e-06 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219967+0000", 2024-09-15T01:32:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023393399999999999 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219983+0000", 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.632e-05 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.220237+0000", 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025403200000000001 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.220252+0000", 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4999e-05 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.220889+0000", 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00063646500000000001 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.221038+0000", 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014901500000000001 2024-09-15T01:32:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.222838+0000", 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018000080000000001 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 39/31 ECSubWrite(tid=40, reqid=client.4477.0:246, at_version=39'8, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.285827+0000", 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.513147569, 2024-09-15T01:32:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019638820000000001, 2024-09-15T01:32:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285827+0000", 2024-09-15T01:32:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285827+0000", 2024-09-15T01:32:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285828+0000", 2024-09-15T01:32:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2200000000000002e-07 2024-09-15T01:32:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285868+0000", 2024-09-15T01:32:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9607000000000002e-05 2024-09-15T01:32:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285895+0000", 2024-09-15T01:32:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7535999999999999e-05 2024-09-15T01:32:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285905+0000", 2024-09-15T01:32:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4020000000000008e-06 2024-09-15T01:32:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285945+0000", 2024-09-15T01:32:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0535999999999999e-05 2024-09-15T01:32:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285954+0000", 2024-09-15T01:32:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8119999999999997e-06 2024-09-15T01:32:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.286127+0000", 2024-09-15T01:32:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000172726 2024-09-15T01:32:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.286136+0000", 2024-09-15T01:32:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4599999999999992e-06 2024-09-15T01:32:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.286174+0000", 2024-09-15T01:32:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7512000000000003e-05 2024-09-15T01:32:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.286188+0000", 2024-09-15T01:32:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4287000000000001e-05 2024-09-15T01:32:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.287791+0000", 2024-09-15T01:32:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001603082 2024-09-15T01:32:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs2 39/38 ECSubWrite(tid=46, reqid=client.4477.0:247, at_version=39'9, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.298214+0000", 2024-09-15T01:32:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.5007610649999998, 2024-09-15T01:32:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018984570000000001, 2024-09-15T01:32:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.298214+0000", 2024-09-15T01:32:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.298214+0000", 2024-09-15T01:32:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.298215+0000", 2024-09-15T01:32:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7300000000000004e-07 2024-09-15T01:32:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.298579+0000", 2024-09-15T01:32:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036413500000000002 2024-09-15T01:32:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.298606+0000", 2024-09-15T01:32:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7305e-05 2024-09-15T01:32:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.298611+0000", 2024-09-15T01:32:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8069999999999999e-06 2024-09-15T01:32:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.298653+0000", 2024-09-15T01:32:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2308000000000002e-05 2024-09-15T01:32:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.298663+0000", 2024-09-15T01:32:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9639999999999998e-06 2024-09-15T01:32:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.298976+0000", 2024-09-15T01:32:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031271499999999998 2024-09-15T01:32:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.298984+0000", 2024-09-15T01:32:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3229999999999998e-06 2024-09-15T01:32:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.299074+0000", 2024-09-15T01:32:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9948e-05 2024-09-15T01:32:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.299090+0000", 2024-09-15T01:32:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5492e-05 2024-09-15T01:32:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.300112+0000", 2024-09-15T01:32:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010224870000000001 2024-09-15T01:32:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 42/31 ECSubWrite(tid=53, reqid=client.4477.0:275, at_version=42'16, trim_to=0'0, roll_forward_to=40'14))", 2024-09-15T01:32:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:44.235412+0000", 2024-09-15T01:32:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.56356242400000001, 2024-09-15T01:32:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034438530000000002, 2024-09-15T01:32:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.235412+0000", 2024-09-15T01:32:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.235412+0000", 2024-09-15T01:32:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.235413+0000", 2024-09-15T01:32:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4999999999999998e-07 2024-09-15T01:32:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.237372+0000", 2024-09-15T01:32:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001958909 2024-09-15T01:32:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.237418+0000", 2024-09-15T01:32:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6409000000000002e-05 2024-09-15T01:32:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.237425+0000", 2024-09-15T01:32:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6490000000000002e-06 2024-09-15T01:32:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.237457+0000", 2024-09-15T01:32:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2824000000000003e-05 2024-09-15T01:32:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.237465+0000", 2024-09-15T01:32:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8229999999999995e-06 2024-09-15T01:32:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.237735+0000", 2024-09-15T01:32:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026918999999999999 2024-09-15T01:32:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.237745+0000", 2024-09-15T01:32:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0657e-05 2024-09-15T01:32:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.237850+0000", 2024-09-15T01:32:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010485899999999999 2024-09-15T01:32:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.237869+0000", 2024-09-15T01:32:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8536999999999999e-05 2024-09-15T01:32:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.238856+0000", 2024-09-15T01:32:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00098764600000000001 2024-09-15T01:32:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 42/31 ECSubWrite(tid=54, reqid=client.4477.0:276, at_version=42'17, trim_to=0'0, roll_forward_to=40'14))", 2024-09-15T01:32:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:44.237375+0000", 2024-09-15T01:32:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.56159996499999998, 2024-09-15T01:32:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00167103, 2024-09-15T01:32:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.237375+0000", 2024-09-15T01:32:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.237375+0000", 2024-09-15T01:32:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.237375+0000", 2024-09-15T01:32:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8100000000000003e-07 2024-09-15T01:32:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.238099+0000", 2024-09-15T01:32:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00072371600000000001 2024-09-15T01:32:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.238137+0000", 2024-09-15T01:32:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8099000000000002e-05 2024-09-15T01:32:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.238147+0000", 2024-09-15T01:32:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4390000000000006e-06 2024-09-15T01:32:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.238202+0000", 2024-09-15T01:32:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5538999999999998e-05 2024-09-15T01:32:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.238212+0000", 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8549999999999997e-06 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.238346+0000", 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013433100000000001 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.238357+0000", 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0841e-05 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.238430+0000", 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2537e-05 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.238490+0000", 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0201000000000002e-05 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:44.239046+0000", 2024-09-15T01:32:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055589099999999998 2024-09-15T01:32:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:44.918 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:44.920 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:32:44.920 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:44.918+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:45.020 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:45.019+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:45.057 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:45.058 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:45.058 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:32:45.058 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:45.066 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:32:45.121 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:45.119+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:45.162 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:32:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:32:45.162 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:45.170 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:32:45.221 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:45.220+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:45.230 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T01:32:45.230 INFO:tasks.rados.rados.0.smithi175.stdout:101: done (1 left) 2024-09-15T01:32:45.230 INFO:tasks.rados.rados.0.smithi175.stdout:102: snap_remove snap 0 2024-09-15T01:32:45.243 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T01:32:45.246 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T01:32:45.270 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:32:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:32:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:32:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:34 3.8s0 3:1c3041d9:::smithi17529310-12:head [append 0~196608,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.185619+0000", 2024-09-15T01:32:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.0814611470000006, 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007701467, 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.185619+0000", 2024-09-15T01:32:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.185619+0000", 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.185620+0000", 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.553e-06 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.185999+0000", 2024-09-15T01:32:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037859699999999999 2024-09-15T01:32:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186023+0000", 2024-09-15T01:32:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3924999999999999e-05 2024-09-15T01:32:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186032+0000", 2024-09-15T01:32:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0750000000000004e-06 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186145+0000", 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011304299999999999 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186295+0000", 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014989800000000001 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187100+0000", 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000805809 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187274+0000", 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000174053 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187351+0000", 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6478999999999997e-05 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187422+0000", 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0727000000000002e-05 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188232+0000", 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00080993799999999996 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.193266+0000", 2024-09-15T01:32:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005034396 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.193320+0000", 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3974000000000002e-05 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:35 3.8s0 3:1c3041d9:::smithi17529310-12:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.186062+0000", 2024-09-15T01:32:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.0810176400000007, 2024-09-15T01:32:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074394359999999998, 2024-09-15T01:32:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35 2024-09-15T01:32:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186062+0000", 2024-09-15T01:32:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186062+0000", 2024-09-15T01:32:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186063+0000", 2024-09-15T01:32:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0720000000000001e-06 2024-09-15T01:32:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186076+0000", 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3057999999999999e-05 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186085+0000", 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3200000000000006e-06 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186112+0000", 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6491000000000001e-05 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187372+0000", 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012596770000000001 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187464+0000", 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1959999999999994e-05 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187885+0000", 2024-09-15T01:32:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00042125099999999998 2024-09-15T01:32:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187986+0000", 2024-09-15T01:32:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010145999999999999 2024-09-15T01:32:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188002+0000", 2024-09-15T01:32:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6082999999999998e-05 2024-09-15T01:32:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188046+0000", 2024-09-15T01:32:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3631999999999998e-05 2024-09-15T01:32:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188236+0000", 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000189994 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.193430+0000", 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051937349999999997 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.193501+0000", 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1703000000000001e-05 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:36 3.8s0 3:1c3041d9:::smithi17529310-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.186121+0000", 2024-09-15T01:32:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.080958335, 2024-09-15T01:32:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0086069019999999996, 2024-09-15T01:32:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:32:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 36 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186121+0000", 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186121+0000", 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186122+0000", 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3200000000000004e-07 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186126+0000", 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.58e-06 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186135+0000", 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8969999999999997e-06 2024-09-15T01:32:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186139+0000", 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8439999999999997e-06 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188124+0000", 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019841070000000001 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188203+0000", 2024-09-15T01:32:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.193776+0000", 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005573153 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.193859+0000", 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2838999999999997e-05 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.194728+0000", 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00086942300000000003 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:94 3.8s0 3:1226c857:::smithi17529310-32:head [append 0~524288,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.231883+0000", 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.0351970260000005, 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029197580000000002, 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 94 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.231883+0000", 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.231883+0000", 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.231884+0000", 2024-09-15T01:32:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5570000000000001e-06 2024-09-15T01:32:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232485+0000", 2024-09-15T01:32:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00060048399999999998 2024-09-15T01:32:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232498+0000", 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3368e-05 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232504+0000", 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4650000000000002e-06 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232544+0000", 2024-09-15T01:32:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0924999999999999e-05 2024-09-15T01:32:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232670+0000", 2024-09-15T01:32:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012542400000000001 2024-09-15T01:32:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.233295+0000", 2024-09-15T01:32:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062494899999999997 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.233412+0000", 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117426 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.233421+0000", 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4999999999999999e-06 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.233510+0000", 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9034000000000002e-05 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.234335+0000", 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00082528099999999997 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.234759+0000", 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00042382100000000001 2024-09-15T01:32:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.234802+0000", 2024-09-15T01:32:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3523999999999999e-05 2024-09-15T01:32:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:95 3.8s0 3:1226c857:::smithi17529310-32:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.232513+0000", 2024-09-15T01:32:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.0345670069999997, 2024-09-15T01:32:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024034429999999999, 2024-09-15T01:32:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 95 2024-09-15T01:32:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232513+0000", 2024-09-15T01:32:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232513+0000", 2024-09-15T01:32:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232513+0000", 2024-09-15T01:32:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8700000000000005e-07 2024-09-15T01:32:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232517+0000", 2024-09-15T01:32:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.642e-06 2024-09-15T01:32:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232524+0000", 2024-09-15T01:32:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-09-15T01:32:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232528+0000", 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0260000000000001e-06 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.233511+0000", 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00098309299999999999 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.233655+0000", 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014415900000000001 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.234037+0000", 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038196299999999997 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.234152+0000", 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115177 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.234158+0000", 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9440000000000002e-06 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.234190+0000", 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1640000000000002e-05 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.234455+0000", 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026545300000000002 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.234869+0000", 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000414349 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.234916+0000", 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6699999999999997e-05 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:96 3.8s0 3:1226c857:::smithi17529310-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.232549+0000", 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.0345307320000003, 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033522349999999999, 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:32:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 96 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232549+0000", 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232549+0000", 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232550+0000", 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8899999999999999e-07 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232553+0000", 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1370000000000002e-06 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232559+0000", 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0850000000000002e-06 2024-09-15T01:32:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.232563+0000", 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0409999999999999e-06 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.234237+0000", 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016737740000000001 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.234325+0000", 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8393999999999994e-05 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.235125+0000", 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00079988699999999999 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.235190+0000", 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4707000000000002e-05 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.235901+0000", 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071162099999999995 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:124 3.8s0 3:1abde0c6:::smithi17529310-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.270268+0000", 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.9968117069999991, 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029772499999999999, 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 124 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270268+0000", 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270268+0000", 2024-09-15T01:32:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270270+0000", 2024-09-15T01:32:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9069999999999999e-06 2024-09-15T01:32:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270961+0000", 2024-09-15T01:32:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00069156100000000004 2024-09-15T01:32:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270983+0000", 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.19e-05 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270993+0000", 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2769999999999994e-06 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.271033+0000", 2024-09-15T01:32:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0581000000000003e-05 2024-09-15T01:32:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.271154+0000", 2024-09-15T01:32:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012120699999999999 2024-09-15T01:32:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.271871+0000", 2024-09-15T01:32:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071659200000000005 2024-09-15T01:32:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.271999+0000", 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012752099999999999 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272007+0000", 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4030000000000007e-06 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272123+0000", 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011577 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.272762+0000", 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00063973700000000003 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273205+0000", 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000442595 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.273245+0000", 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0198999999999999e-05 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.3s5 34/31 ECSubWrite(tid=136, reqid=client.4477.0:154, at_version=34'9, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.308439+0000", 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.9586409499999995, 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059934430000000002, 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308439+0000", 2024-09-15T01:32:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308439+0000", 2024-09-15T01:32:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308440+0000", 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9899999999999999e-07 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308458+0000", 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8005000000000001e-05 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308504+0000", 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5989000000000003e-05 2024-09-15T01:32:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308510+0000", 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5509999999999996e-06 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308565+0000", 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4715e-05 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308576+0000", 2024-09-15T01:32:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0913e-05 2024-09-15T01:32:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308633+0000", 2024-09-15T01:32:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7691e-05 2024-09-15T01:32:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308641+0000", 2024-09-15T01:32:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5329999999999997e-06 2024-09-15T01:32:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308687+0000", 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5589e-05 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308710+0000", 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3121000000000001e-05 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.314432+0000", 2024-09-15T01:32:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0057225369999999998 2024-09-15T01:32:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.3s5 34/31 ECSubWrite(tid=137, reqid=client.4477.0:154, at_version=0'0, trim_to=0'0, roll_forward_to=34'9))", 2024-09-15T01:32:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.310858+0000", 2024-09-15T01:32:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.9562219970000001, 2024-09-15T01:32:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037232379999999998, 2024-09-15T01:32:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.310858+0000", 2024-09-15T01:32:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.310858+0000", 2024-09-15T01:32:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.310858+0000", 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2900000000000003e-07 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.310868+0000", 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8030000000000008e-06 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.310903+0000", 2024-09-15T01:32:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4292999999999998e-05 2024-09-15T01:32:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.310909+0000", 2024-09-15T01:32:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5039999999999999e-06 2024-09-15T01:32:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.310954+0000", 2024-09-15T01:32:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5201999999999999e-05 2024-09-15T01:32:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.310964+0000", 2024-09-15T01:32:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6670000000000008e-06 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.311033+0000", 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9457999999999998e-05 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.311041+0000", 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.576e-06 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.311546+0000", 2024-09-15T01:32:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00050557 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.311576+0000", 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9694999999999999e-05 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.314581+0000", 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003004741 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:206 3.8s0 3:1226c857:::smithi17529310-32:head [append 0~589824] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.217163+0000", 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0499163749999996, 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050434260000000002, 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 206 2024-09-15T01:32:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.217163+0000", 2024-09-15T01:32:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.217163+0000", 2024-09-15T01:32:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.217165+0000", 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5939999999999999e-06 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.218264+0000", 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001098924 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.218282+0000", 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7918e-05 2024-09-15T01:32:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.218288+0000", 2024-09-15T01:32:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4119999999999998e-06 2024-09-15T01:32:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.218334+0000", 2024-09-15T01:32:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5454000000000002e-05 2024-09-15T01:32:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.218410+0000", 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6203000000000002e-05 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219215+0000", 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00080522199999999997 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219458+0000", 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024316799999999999 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219507+0000", 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9268999999999999e-05 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219661+0000", 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015368399999999999 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219681+0000", 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0275000000000002e-05 2024-09-15T01:32:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221843+0000", 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002161662 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.222207+0000", 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000363641 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:207 3.8s0 3:1226c857:::smithi17529310-32:head [append 0~376832] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:32:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.219252+0000", 2024-09-15T01:32:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0478278100000002, 2024-09-15T01:32:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033414920000000002, 2024-09-15T01:32:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 207 2024-09-15T01:32:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219252+0000", 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219252+0000", 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219253+0000", 2024-09-15T01:32:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.271e-06 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219837+0000", 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00058401799999999995 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219854+0000", 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6464e-05 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219860+0000", 2024-09-15T01:32:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8390000000000001e-06 2024-09-15T01:32:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219907+0000", 2024-09-15T01:32:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6978999999999999e-05 2024-09-15T01:32:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220026+0000", 2024-09-15T01:32:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118615 2024-09-15T01:32:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220653+0000", 2024-09-15T01:32:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062735299999999998 2024-09-15T01:32:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220769+0000", 2024-09-15T01:32:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115293 2024-09-15T01:32:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220778+0000", 2024-09-15T01:32:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8659999999999995e-06 2024-09-15T01:32:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220874+0000", 2024-09-15T01:32:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6058999999999996e-05 2024-09-15T01:32:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221489+0000", 2024-09-15T01:32:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00061580599999999999 2024-09-15T01:32:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.222270+0000", 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00078016499999999998 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.222593+0000", 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032376399999999999 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:208 3.8s0 3:1226c857:::smithi17529310-32:head [setxattr _header (56) in=7b] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.219870+0000", 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0472096369999999, 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93580432499999999, 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 208 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219870+0000", 2024-09-15T01:32:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219870+0000", 2024-09-15T01:32:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219871+0000", 2024-09-15T01:32:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9100000000000002e-07 2024-09-15T01:32:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219875+0000", 2024-09-15T01:32:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7340000000000002e-06 2024-09-15T01:32:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219882+0000", 2024-09-15T01:32:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1729999999999998e-06 2024-09-15T01:32:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219886+0000", 2024-09-15T01:32:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5129999999999998e-06 2024-09-15T01:32:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220853+0000", 2024-09-15T01:32:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096616900000000001 2024-09-15T01:32:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220951+0000", 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8140000000000006e-05 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221294+0000", 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034291600000000002 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221386+0000", 2024-09-15T01:32:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2517999999999997e-05 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221394+0000", 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8299999999999996e-06 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.223718+0000", 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023245449999999999 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.223730+0000", 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1659e-05 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.155614+0000", 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93188401099999996 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.155674+0000", 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0226000000000002e-05 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:209 3.8s0 3:1226c857:::smithi17529310-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.219933+0000", 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0471463910000001, 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.937063969, 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 209 2024-09-15T01:32:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219933+0000", 2024-09-15T01:32:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219933+0000", 2024-09-15T01:32:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219934+0000", 2024-09-15T01:32:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999999e-07 2024-09-15T01:32:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219937+0000", 2024-09-15T01:32:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3230000000000002e-06 2024-09-15T01:32:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219949+0000", 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1205e-05 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219956+0000", 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1829999999999997e-06 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.222023+0000", 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020672009999999998 2024-09-15T01:32:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.222072+0000", 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8752999999999999e-05 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.155960+0000", 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93388811299999996 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.156034+0000", 2024-09-15T01:32:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4264999999999995e-05 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.156997+0000", 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096320599999999998 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.3s5 39/31 ECSubWrite(tid=144, reqid=client.4477.0:222, at_version=39'11, trim_to=0'0, roll_forward_to=34'9))", 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.218520+0000", 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0485595950000004, 2024-09-15T01:32:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002772615, 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.218520+0000", 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.218520+0000", 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.218520+0000", 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9299999999999999e-07 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.218673+0000", 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000152497 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.218727+0000", 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4138999999999998e-05 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.218737+0000", 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0173000000000001e-05 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.218793+0000", 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5399000000000003e-05 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.218800+0000", 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.934e-06 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219171+0000", 2024-09-15T01:32:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037134400000000001 2024-09-15T01:32:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219184+0000", 2024-09-15T01:32:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3606e-05 2024-09-15T01:32:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219288+0000", 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010333299999999999 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219309+0000", 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1203999999999999e-05 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.221293+0000", 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019836929999999999 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:233 3.8s0 3:1226c857:::smithi17529310-32:head [append 0~688128] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.262917+0000", 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0041630399999999, 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00455018, 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 233 2024-09-15T01:32:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.262917+0000", 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.262917+0000", 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.262919+0000", 2024-09-15T01:32:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.263608+0000", 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068956200000000001 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.263623+0000", 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4633e-05 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.263628+0000", 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3229999999999997e-06 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.263677+0000", 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8627e-05 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.263746+0000", 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9278999999999999e-05 2024-09-15T01:32:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.264493+0000", 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00074680799999999996 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.264594+0000", 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101216 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.264604+0000", 2024-09-15T01:32:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3889999999999994e-06 2024-09-15T01:32:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.264712+0000", 2024-09-15T01:32:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000108206 2024-09-15T01:32:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.264739+0000", 2024-09-15T01:32:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6823000000000001e-05 2024-09-15T01:32:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.267406+0000", 2024-09-15T01:32:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026674440000000002 2024-09-15T01:32:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.267467+0000", 2024-09-15T01:32:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0807000000000002e-05 2024-09-15T01:32:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:234 3.8s0 3:1226c857:::smithi17529310-32:head [append 0~376832] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.265009+0000", 2024-09-15T01:32:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0020705339999996, 2024-09-15T01:32:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028683279999999998, 2024-09-15T01:32:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 234 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265009+0000", 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265009+0000", 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265011+0000", 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3570000000000001e-06 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265448+0000", 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00043753100000000001 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265462+0000", 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4265e-05 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265470+0000", 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.977e-06 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265520+0000", 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9817000000000001e-05 2024-09-15T01:32:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265633+0000", 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112784 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.266223+0000", 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00059046299999999995 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.266330+0000", 2024-09-15T01:32:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010671999999999999 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.266342+0000", 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2207e-05 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.266424+0000", 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1664999999999998e-05 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.267183+0000", 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075914600000000004 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.267815+0000", 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00063216399999999995 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.267877+0000", 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2231999999999996e-05 2024-09-15T01:32:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:235 3.8s0 3:1226c857:::smithi17529310-32:head [setxattr _header (56) in=7b] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.265489+0000", 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0015910029999997, 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027079579999999999, 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 235 2024-09-15T01:32:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265489+0000", 2024-09-15T01:32:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265489+0000", 2024-09-15T01:32:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265489+0000", 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265493+0000", 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9129999999999996e-06 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265502+0000", 2024-09-15T01:32:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2800000000000003e-06 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265507+0000", 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7130000000000002e-06 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.266417+0000", 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00090956800000000003 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.266523+0000", 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010589100000000001 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.266996+0000", 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000473175 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.267100+0000", 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104337 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.267110+0000", 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6739999999999992e-06 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.267153+0000", 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2883000000000003e-05 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.267219+0000", 2024-09-15T01:32:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6351000000000002e-05 2024-09-15T01:32:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.268141+0000", 2024-09-15T01:32:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092225499999999997 2024-09-15T01:32:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.268197+0000", 2024-09-15T01:32:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5201999999999998e-05 2024-09-15T01:32:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:236 3.8s0 3:1226c857:::smithi17529310-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.265524+0000", 2024-09-15T01:32:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0015555440000004, 2024-09-15T01:32:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038214690000000001, 2024-09-15T01:32:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:32:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 236 2024-09-15T01:32:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265524+0000", 2024-09-15T01:32:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265524+0000", 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265525+0000", 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6199999999999998e-07 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265529+0000", 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-09-15T01:32:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265536+0000", 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8090000000000004e-06 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.265539+0000", 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3440000000000001e-06 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.268496+0000", 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029561370000000002 2024-09-15T01:32:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.268589+0000", 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3135999999999997e-05 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.269346+0000", 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075695500000000002 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs6 39/31 ECSubWrite(tid=58, reqid=client.4477.0:240, at_version=39'20, trim_to=0'0, roll_forward_to=39'18))", 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.279584+0000", 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9874957210000002, 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024461359999999998, 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.279584+0000", 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.279584+0000", 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.279585+0000", 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6300000000000004e-07 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.280302+0000", 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071749900000000002 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.280329+0000", 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6755999999999999e-05 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.280334+0000", 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0130000000000001e-06 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.280389+0000", 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5056999999999998e-05 2024-09-15T01:32:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.280400+0000", 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0892e-05 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.280666+0000", 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026559499999999998 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.280699+0000", 2024-09-15T01:32:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3139999999999998e-05 2024-09-15T01:32:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.280794+0000", 2024-09-15T01:32:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5376000000000003e-05 2024-09-15T01:32:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.280810+0000", 2024-09-15T01:32:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5701000000000001e-05 2024-09-15T01:32:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.282030+0000", 2024-09-15T01:32:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012202440000000001 2024-09-15T01:32:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s4 39/31 ECSubWrite(tid=40, reqid=client.4477.0:246, at_version=39'8, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:32:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.285905+0000", 2024-09-15T01:32:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9811742790000002, 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002374924, 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285905+0000", 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285905+0000", 2024-09-15T01:32:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285906+0000", 2024-09-15T01:32:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5099999999999999e-07 2024-09-15T01:32:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285924+0000", 2024-09-15T01:32:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8130000000000001e-05 2024-09-15T01:32:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285954+0000", 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0178000000000001e-05 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285960+0000", 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4820000000000002e-06 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.285999+0000", 2024-09-15T01:32:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8952999999999998e-05 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.286006+0000", 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4900000000000003e-06 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.286157+0000", 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015101700000000001 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.286169+0000", 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1123e-05 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.286207+0000", 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8781e-05 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.286221+0000", 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4045e-05 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.288280+0000", 2024-09-15T01:32:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020589739999999999 2024-09-15T01:32:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.399 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:45.401 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:32:45.402 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:45.399+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:45.500 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:45.500 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:45.500 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:32:45.501 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:45.502 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:45.500+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:45.509 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:32:45.602 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:45.601+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:45.604 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:32:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:32:45.605 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:45.613 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:32:45.704 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:45.702+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 33/31 ECSubWrite(tid=2, reqid=client.4477.0:1, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.176343+0000", 2024-09-15T01:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5729464520000001, 2024-09-15T01:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096529600000000004, 2024-09-15T01:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176343+0000", 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176343+0000", 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176343+0000", 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3999999999999998e-07 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176405+0000", 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1896000000000005e-05 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176489+0000", 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3608000000000003e-05 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176500+0000", 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0685e-05 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176570+0000", 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0057999999999999e-05 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176580+0000", 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0309000000000001e-05 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176704+0000", 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012443600000000001 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176750+0000", 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5213999999999998e-05 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176809+0000", 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9477e-05 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176988+0000", 2024-09-15T01:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178636 2024-09-15T01:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177309+0000", 2024-09-15T01:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032073699999999998 2024-09-15T01:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 33/31 ECSubWrite(tid=3, reqid=client.4477.0:2, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.176414+0000", 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5728759070000002, 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010880270000000001, 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176414+0000", 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176414+0000", 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176414+0000", 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7100000000000002e-07 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176430+0000", 2024-09-15T01:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5682e-05 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176522+0000", 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2359000000000001e-05 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176527+0000", 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176766+0000", 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000238411 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176802+0000", 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5769000000000003e-05 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176941+0000", 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013990500000000001 2024-09-15T01:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176969+0000", 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7251999999999999e-05 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177029+0000", 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9933999999999999e-05 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177048+0000", 2024-09-15T01:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9123e-05 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177502+0000", 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045398500000000001 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 34/31 ECSubWrite(tid=0, reqid=client.4477.0:13, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179137+0000", 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5701523430000002, 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017856180000000001, 2024-09-15T01:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179137+0000", 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179137+0000", 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179138+0000", 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3900000000000004e-07 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179179+0000", 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0599999999999998e-05 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179867+0000", 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068798899999999996 2024-09-15T01:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179876+0000", 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180320+0000", 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044354399999999998 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180328+0000", 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7649999999999993e-06 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180424+0000", 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6311000000000007e-05 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180437+0000", 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3001e-05 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180528+0000", 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1378000000000005e-05 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180623+0000", 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4455000000000003e-05 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180923+0000", 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030002300000000001 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=1, reqid=client.4477.0:28, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5701071310000003, 2024-09-15T01:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018499090000000001, 2024-09-15T01:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179245+0000", 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2150000000000006e-05 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180346+0000", 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001101196 2024-09-15T01:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180357+0000", 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175000000000001e-05 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180408+0000", 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0590999999999997e-05 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180417+0000", 2024-09-15T01:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8060000000000001e-06 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180553+0000", 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136324 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180684+0000", 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000130853 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180745+0000", 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0971000000000003e-05 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180919+0000", 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017386800000000001 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181032+0000", 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113578 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 34/31 ECSubWrite(tid=2, reqid=client.4477.0:14, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5700407960000007, 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018048179999999999, 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179259+0000", 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0104e-05 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180590+0000", 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013307779999999999 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180598+0000", 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-15T01:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180692+0000", 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3549999999999997e-05 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180700+0000", 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6330000000000002e-06 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180793+0000", 2024-09-15T01:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967000000000006e-05 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180801+0000", 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4580000000000004e-06 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180836+0000", 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5732999999999998e-05 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180858+0000", 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.126e-05 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181054+0000", 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019604099999999999 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 33/31 ECSubWrite(tid=4, reqid=client.4477.0:2, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179837+0000", 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.569452471, 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00111943, 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179837+0000", 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179837+0000", 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179838+0000", 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.096e-06 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179853+0000", 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4919999999999999e-05 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179893+0000", 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9502999999999998e-05 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179902+0000", 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179998+0000", 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6417999999999997e-05 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180008+0000", 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8980000000000008e-06 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180107+0000", 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9267999999999999e-05 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180121+0000", 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3715e-05 2024-09-15T01:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180201+0000", 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0383999999999995e-05 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180934+0000", 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00073209700000000002 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180957+0000", 2024-09-15T01:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3104000000000001e-05 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 34/31 ECSubWrite(tid=4, reqid=client.4477.0:31, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.180672+0000", 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5686173869999998, 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014757450000000001, 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180672+0000", 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180672+0000", 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180673+0000", 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2799999999999996e-07 2024-09-15T01:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181385+0000", 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071188100000000004 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181419+0000", 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4078999999999997e-05 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181425+0000", 2024-09-15T01:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5989999999999998e-06 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181488+0000", 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2407000000000001e-05 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181498+0000", 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003999999999999e-05 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181638+0000", 2024-09-15T01:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014010499999999999 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181679+0000", 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0672e-05 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181759+0000", 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0067000000000004e-05 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181779+0000", 2024-09-15T01:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0336e-05 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182148+0000", 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000369067 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=0, reqid=client.4477.0:16, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.181909+0000", 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5673809809999995, 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00076923200000000001, 2024-09-15T01:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181909+0000", 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181909+0000", 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181909+0000", 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2500000000000001e-07 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181978+0000", 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8928000000000001e-05 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182070+0000", 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2204e-05 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182077+0000", 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0210000000000002e-06 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182122+0000", 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4582000000000001e-05 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182131+0000", 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8510000000000005e-06 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182261+0000", 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000129875 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182294+0000", 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3291999999999999e-05 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182356+0000", 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1696e-05 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182487+0000", 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013164000000000001 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182678+0000", 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019071800000000001 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=1, reqid=client.4477.0:17, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.183115+0000", 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5661745909999993, 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00064692799999999996, 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183115+0000", 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183115+0000", 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183116+0000", 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000003e-07 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183134+0000", 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8031000000000001e-05 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183179+0000", 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5466999999999997e-05 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183187+0000", 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1899999999999995e-06 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183249+0000", 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1975000000000002e-05 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183276+0000", 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7056e-05 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183365+0000", 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8863000000000006e-05 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183397+0000", 2024-09-15T01:32:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2082999999999998e-05 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183439+0000", 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1269000000000001e-05 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183458+0000", 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9570000000000001e-05 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183762+0000", 2024-09-15T01:32:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030372599999999999 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 34/31 ECSubWrite(tid=6, reqid=client.4477.0:43, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.187004+0000", 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5622858110000006, 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011234019999999999, 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187004+0000", 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187004+0000", 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187005+0000", 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0470000000000001e-06 2024-09-15T01:32:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187114+0000", 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010932700000000001 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187177+0000", 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2597999999999999e-05 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187185+0000", 2024-09-15T01:32:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1780000000000003e-06 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187279+0000", 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3690999999999995e-05 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187288+0000", 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2799999999999992e-06 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187629+0000", 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034080200000000002 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187682+0000", 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2848e-05 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187752+0000", 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9906000000000005e-05 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188095+0000", 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000343297 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188127+0000", 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2428000000000002e-05 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=2, reqid=client.4477.0:17, at_version=0'0, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.187733+0000", 2024-09-15T01:32:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5615562880000002, 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010603889999999999, 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187733+0000", 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187733+0000", 2024-09-15T01:32:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188139+0000", 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040583200000000001 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188168+0000", 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8878999999999999e-05 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188207+0000", 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9385999999999999e-05 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188219+0000", 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1521e-05 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188274+0000", 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5324000000000001e-05 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188284+0000", 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8239999999999995e-06 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188391+0000", 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106909 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188449+0000", 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8193999999999999e-05 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188505+0000", 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6033999999999999e-05 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188615+0000", 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109959 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188794+0000", 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017852699999999999 2024-09-15T01:32:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=16, reqid=client.4477.0:76, at_version=34'5, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.214947+0000", 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5343427460000001, 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071814299999999997, 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.214947+0000", 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.214947+0000", 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.214948+0000", 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4810000000000001e-06 2024-09-15T01:32:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215065+0000", 2024-09-15T01:32:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116551 2024-09-15T01:32:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215104+0000", 2024-09-15T01:32:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9462000000000003e-05 2024-09-15T01:32:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215113+0000", 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2689999999999999e-06 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215157+0000", 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4127e-05 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215163+0000", 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4670000000000001e-06 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215295+0000", 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013140999999999999 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215304+0000", 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0399999999999998e-06 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215379+0000", 2024-09-15T01:32:45.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5007999999999997e-05 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215395+0000", 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5897999999999999e-05 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215665+0000", 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027043 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=4, reqid=client.4477.0:88, at_version=34'3, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.226933+0000", 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5223568620000005, 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010392190000000001, 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.226933+0000", 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.226933+0000", 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.226934+0000", 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.077e-06 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227240+0000", 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030573500000000001 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227270+0000", 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.021e-05 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227277+0000", 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1629999999999999e-06 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227326+0000", 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9227000000000001e-05 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227335+0000", 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9609999999999994e-06 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227508+0000", 2024-09-15T01:32:45.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017294000000000001 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227520+0000", 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1963e-05 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227623+0000", 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102663 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227724+0000", 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010162600000000001 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227972+0000", 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024765399999999998 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=5, reqid=client.4477.0:89, at_version=34'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:32:45.824 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.227505+0000", 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5217849999999995, 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00064992299999999995, 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227505+0000", 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227505+0000", 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227505+0000", 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1800000000000005e-07 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227523+0000", 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7708999999999999e-05 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227558+0000", 2024-09-15T01:32:45.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4310000000000002e-05 2024-09-15T01:32:45.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227564+0000", 2024-09-15T01:32:45.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4690000000000002e-06 2024-09-15T01:32:45.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227610+0000", 2024-09-15T01:32:45.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6440000000000003e-05 2024-09-15T01:32:45.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227620+0000", 2024-09-15T01:32:45.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4029999999999996e-06 2024-09-15T01:32:45.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227705+0000", 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5518000000000004e-05 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227713+0000", 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.627e-06 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227739+0000", 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6342999999999998e-05 2024-09-15T01:32:45.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227755+0000", 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6164e-05 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.228155+0000", 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039912200000000002 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=18, reqid=client.4477.0:91, at_version=34'7, trim_to=0'0, roll_forward_to=34'5))", 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.230472+0000", 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5188180710000001, 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00078053499999999995, 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230472+0000", 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230472+0000", 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230472+0000", 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7800000000000001e-07 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230677+0000", 2024-09-15T01:32:45.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000204157 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230711+0000", 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4150999999999999e-05 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230718+0000", 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4869999999999996e-06 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230760+0000", 2024-09-15T01:32:45.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1587e-05 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230768+0000", 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1640000000000001e-06 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230905+0000", 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000137269 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230916+0000", 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1104000000000001e-05 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.231004+0000", 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7379999999999993e-05 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.231016+0000", 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1934e-05 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.231252+0000", 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236524 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 34/31 ECSubWrite(tid=6, reqid=client.4477.0:118, at_version=34'3, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.263014+0000", 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.4862754230000004, 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001181788, 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263014+0000", 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263014+0000", 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263016+0000", 2024-09-15T01:32:45.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.424e-06 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263549+0000", 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00053332100000000001 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263582+0000", 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2926000000000003e-05 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263591+0000", 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263633+0000", 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2271999999999998e-05 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263640+0000", 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8190000000000003e-06 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263822+0000", 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018233 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263831+0000", 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263930+0000", 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8944999999999995e-05 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263964+0000", 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4173e-05 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.264196+0000", 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023164099999999999 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 34/31 ECSubWrite(tid=7, reqid=client.4477.0:119, at_version=34'4, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:32:45.841 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.263660+0000", 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.4856298339999991, 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065152199999999997, 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263660+0000", 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263660+0000", 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263660+0000", 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0400000000000002e-07 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263674+0000", 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3579e-05 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263697+0000", 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3499999999999999e-05 2024-09-15T01:32:45.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263703+0000", 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263869+0000", 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016557099999999999 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263878+0000", 2024-09-15T01:32:45.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263950+0000", 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496999999999994e-05 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263956+0000", 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.305e-06 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263980+0000", 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4036000000000002e-05 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263991+0000", 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1770000000000001e-05 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.264311+0000", 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031986800000000001 2024-09-15T01:32:45.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 34/31 ECSubWrite(tid=23, reqid=client.4477.0:150, at_version=34'3, trim_to=0'0, roll_forward_to=34'1))", 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.307830+0000", 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.4414597550000003, 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070952099999999996, 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307830+0000", 2024-09-15T01:32:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307830+0000", 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307832+0000", 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2900000000000001e-06 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307956+0000", 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123354 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307991+0000", 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5608000000000002e-05 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307999+0000", 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0790000000000001e-06 2024-09-15T01:32:45.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308047+0000", 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7756000000000002e-05 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308054+0000", 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5050000000000004e-06 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:45.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308144+0000", 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0972999999999998e-05 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308152+0000", 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308239+0000", 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6976000000000002e-05 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308258+0000", 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8961e-05 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308539+0000", 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028192700000000001 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.bs6 37/36 ECSubRead(tid=38, to_read={3:df51d772:::smithi17529310-50:head=0,262144,0}, subchunks={3:df51d772:::smithi17529310-50:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:32:45.852 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:39.198209+0000", 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.551080733, 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092439400000000004, 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198209+0000", 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198209+0000", 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198209+0000", 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2e-07 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198215+0000", 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3499999999999996e-06 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198235+0000", 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0565e-05 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198240+0000", 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0540000000000002e-06 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198279+0000", 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8664000000000001e-05 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.199133+0000", 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000854341 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.6s6 37/36 ECSubRead(tid=10, to_read={3:65d6c4b7:::smithi17529310-30:head=0,262144,0}, subchunks={3:65d6c4b7:::smithi17529310-30:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:39.201538+0000", 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.5477511780000004, 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011658199999999999, 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:45.856 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201538+0000", 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201538+0000", 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201539+0000", 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:32:45.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201546+0000", 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5699999999999998e-06 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201571+0000", 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5381e-05 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201576+0000", 2024-09-15T01:32:45.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7790000000000003e-06 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201618+0000", 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1999999999999998e-05 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.202704+0000", 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010864640000000001 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:45.859 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:45.860 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:45.861 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-15T01:32:45.861 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:45.859+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:45.962 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:45.960+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:45.999 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:45.999 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:45.999 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:32:45.999 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:46.007 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-15T01:32:46.063 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:46.061+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:46.104 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:32:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:32:46.104 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:46.112 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-15T01:32:46.164 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:46.162+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:46.211 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:46.211 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:32:46.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:32:46.211 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:32:46.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.211 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:46.211 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:32:46.211 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.728651788000001, 2024-09-15T01:32:46.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27210968699999999, 2024-09-15T01:32:46.211 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.211 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479166+0000", 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4729999999999999e-06 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479172+0000", 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7460000000000003e-06 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479186+0000", 2024-09-15T01:32:46.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4011e-05 2024-09-15T01:32:46.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479211+0000", 2024-09-15T01:32:46.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4658e-05 2024-09-15T01:32:46.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479249+0000", 2024-09-15T01:32:46.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7706000000000001e-05 2024-09-15T01:32:46.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-15T01:32:46.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479255+0000", 2024-09-15T01:32:46.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9669999999999998e-06 2024-09-15T01:32:46.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750042+0000", 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27078760899999998 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750204+0000", 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016171800000000001 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750337+0000", 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000132674 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750470+0000", 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013314699999999999 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750485+0000", 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5288e-05 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750515+0000", 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0179e-05 2024-09-15T01:32:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750582+0000", 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6890999999999994e-05 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751212+0000", 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062971900000000005 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751249+0000", 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7072e-05 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751274+0000", 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4828999999999999e-05 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269156+0000", 2024-09-15T01:32:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.938660249000002, 2024-09-15T01:32:46.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010045861, 2024-09-15T01:32:46.220 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.220 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:46.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:46.220 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 72 2024-09-15T01:32:46.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.220 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269156+0000", 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269156+0000", 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269156+0000", 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1199999999999999e-07 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269218+0000", 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.19e-05 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269222+0000", 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.673e-06 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269225+0000", 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.441e-06 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271274+0000", 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020488540000000001 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271323+0000", 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9258999999999997e-05 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271394+0000", 2024-09-15T01:32:46.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0745000000000001e-05 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271483+0000", 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9605999999999996e-05 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271491+0000", 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5800000000000003e-06 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271554+0000", 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3157000000000005e-05 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271765+0000", 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021087699999999999 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279154+0000", 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073888729999999998 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279181+0000", 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7370000000000001e-05 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279202+0000", 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0313999999999999e-05 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269228+0000", 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.938587901999998, 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010072035, 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 73 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269228+0000", 2024-09-15T01:32:46.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269228+0000", 2024-09-15T01:32:46.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269228+0000", 2024-09-15T01:32:46.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3099999999999999e-07 2024-09-15T01:32:46.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269230+0000", 2024-09-15T01:32:46.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.137e-06 2024-09-15T01:32:46.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269234+0000", 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0599999999999999e-06 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269235+0000", 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.922e-06 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271530+0000", 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0022941419999999999 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271573+0000", 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3789999999999999e-05 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271653+0000", 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9827000000000007e-05 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271737+0000", 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3950999999999996e-05 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271742+0000", 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2449999999999998e-06 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271768+0000", 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5581000000000001e-05 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.271922+0000", 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153871 2024-09-15T01:32:46.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279240+0000", 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073177279999999999 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279275+0000", 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5179999999999999e-05 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279300+0000", 2024-09-15T01:32:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.527e-05 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269238+0000", 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.938578070999998, 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010290084999999999, 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:46.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 74 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269238+0000", 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269238+0000", 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269238+0000", 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269240+0000", 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4310000000000001e-06 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269243+0000", 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-15T01:32:46.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269245+0000", 2024-09-15T01:32:46.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8679999999999999e-06 2024-09-15T01:32:46.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272014+0000", 2024-09-15T01:32:46.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027686450000000001 2024-09-15T01:32:46.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272069+0000", 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5031000000000002e-05 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272175+0000", 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106455 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272298+0000", 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122891 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272304+0000", 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3510000000000001e-06 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272329+0000", 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4905999999999998e-05 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272892+0000", 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00056393100000000005 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279488+0000", 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065956269999999997 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279511+0000", 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2527e-05 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279528+0000", 2024-09-15T01:32:46.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7294000000000001e-05 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269248+0000", 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.938568281999999, 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010540288, 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:46.241 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 75 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269248+0000", 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269248+0000", 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269248+0000", 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.04e-07 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269301+0000", 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2757000000000003e-05 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269305+0000", 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3780000000000003e-06 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269309+0000", 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.738e-06 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272346+0000", 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030372089999999999 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272414+0000", 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8232000000000007e-05 2024-09-15T01:32:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272543+0000", 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012883799999999999 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272646+0000", 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102948 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272651+0000", 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6879999999999998e-06 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272703+0000", 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1901000000000003e-05 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272897+0000", 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000194472 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279745+0000", 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068477929999999996 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279770+0000", 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4820999999999998e-05 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279788+0000", 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8309e-05 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.247 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.248 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:46.248 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.269311+0000", 2024-09-15T01:32:46.248 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.938504449, 2024-09-15T01:32:46.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010539287, 2024-09-15T01:32:46.248 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.248 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:46.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 76 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269311+0000", 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269311+0000", 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269312+0000", 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9799999999999999e-07 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269315+0000", 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7889999999999999e-06 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269318+0000", 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0070000000000002e-06 2024-09-15T01:32:46.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.269320+0000", 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0540000000000002e-06 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272676+0000", 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033561340000000002 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272730+0000", 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3983999999999997e-05 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272797+0000", 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7156999999999994e-05 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272866+0000", 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9453000000000001e-05 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272872+0000", 2024-09-15T01:32:46.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9660000000000001e-06 2024-09-15T01:32:46.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272895+0000", 2024-09-15T01:32:46.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2764000000000001e-05 2024-09-15T01:32:46.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:46.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.272915+0000", 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0361e-05 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279812+0000", 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068969000000000001 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279833+0000", 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1055000000000001e-05 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.279851+0000", 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7365000000000001e-05 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:78 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.280120+0000", 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.927696257000001, 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082697889999999996, 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.255 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280120+0000", 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280120+0000", 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280120+0000", 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280162+0000", 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1817999999999998e-05 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280167+0000", 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280170+0000", 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5310000000000002e-06 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280390+0000", 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021957699999999999 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280462+0000", 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2725000000000006e-05 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280551+0000", 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8246999999999997e-05 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280646+0000", 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5063999999999996e-05 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280651+0000", 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280706+0000", 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4815000000000003e-05 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281099+0000", 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000392982 2024-09-15T01:32:46.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288289+0000", 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071901609999999996 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288336+0000", 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7593999999999999e-05 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288389+0000", 2024-09-15T01:32:46.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3022999999999997e-05 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.280175+0000", 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.927640847999999, 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082971350000000006, 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:46.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 79 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280175+0000", 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280175+0000", 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280175+0000", 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280217+0000", 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1483000000000002e-05 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280222+0000", 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6260000000000003e-06 2024-09-15T01:32:46.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280224+0000", 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5950000000000001e-06 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280700+0000", 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047627200000000002 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280804+0000", 2024-09-15T01:32:46.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103742 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280917+0000", 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112908 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281046+0000", 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000128555 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281059+0000", 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.362e-05 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281110+0000", 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0457000000000002e-05 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281671+0000", 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00056108199999999997 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288422+0000", 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00675145 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288451+0000", 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9133e-05 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288472+0000", 2024-09-15T01:32:46.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0761999999999999e-05 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.280228+0000", 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.927588311000001, 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085368890000000006, 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:46.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:46.270 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 80 2024-09-15T01:32:46.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.270 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280228+0000", 2024-09-15T01:32:46.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280228+0000", 2024-09-15T01:32:46.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280228+0000", 2024-09-15T01:32:46.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8800000000000003e-07 2024-09-15T01:32:46.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280281+0000", 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2437e-05 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280286+0000", 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.835e-06 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.280289+0000", 2024-09-15T01:32:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0790000000000001e-06 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281125+0000", 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00083510599999999996 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281204+0000", 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9025999999999999e-05 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:46.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281350+0000", 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000146866 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281512+0000", 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000161359 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281518+0000", 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1229999999999997e-06 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281573+0000", 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.541e-05 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.281675+0000", 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101724 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288695+0000", 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070196870000000001 2024-09-15T01:32:46.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288734+0000", 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9193000000000002e-05 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.288764+0000", 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0555999999999997e-05 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:100 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:17.395756+0000", 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.812059608999999, 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0055617000000000002, 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 100 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.277 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395756+0000", 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395756+0000", 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395757+0000", 2024-09-15T01:32:46.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395759+0000", 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2309999999999999e-06 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395762+0000", 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9500000000000001e-06 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.395764+0000", 2024-09-15T01:32:46.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9580000000000001e-06 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400609+0000", 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048449870000000003 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400678+0000", 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9764999999999994e-05 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400806+0000", 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012723899999999999 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400928+0000", 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012266800000000001 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400935+0000", 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4570000000000002e-06 2024-09-15T01:32:46.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.400962+0000", 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7223e-05 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401017+0000", 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4868999999999999e-05 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:46.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401246+0000", 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022941300000000001 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401287+0000", 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0193e-05 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:17.401318+0000", 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1452000000000003e-05 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:145 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~28] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:19.474868+0000", 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.732947816999999, 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071687239999999996, 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145 2024-09-15T01:32:46.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474868+0000", 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474868+0000", 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474869+0000", 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3399999999999999e-07 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474873+0000", 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474883+0000", 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0115000000000001e-05 2024-09-15T01:32:46.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474887+0000", 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7000000000000002e-06 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474910+0000", 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099e-05 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.474961+0000", 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1285000000000001e-05 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475027+0000", 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5427999999999998e-05 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475105+0000", 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8459000000000002e-05 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475133+0000", 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8371999999999999e-05 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475163+0000", 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9181999999999999e-05 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.475179+0000", 2024-09-15T01:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6069000000000001e-05 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.481979+0000", 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00680038 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.482007+0000", 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7648e-05 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:19.482037+0000", 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0048000000000001e-05 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:7 3.3s0 3:c07cd22b:::smithi17529310-3:head [append 0~49152,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.175606+0000", 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.032209435, 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011071667, 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:46.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175606+0000", 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175606+0000", 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175610+0000", 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6239999999999999e-06 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175737+0000", 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012691699999999999 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175767+0000", 2024-09-15T01:32:46.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0029999999999999e-05 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175781+0000", 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4173e-05 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175903+0000", 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122031 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176049+0000", 2024-09-15T01:32:46.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014554100000000001 2024-09-15T01:32:46.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176870+0000", 2024-09-15T01:32:46.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00082102800000000001 2024-09-15T01:32:46.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177006+0000", 2024-09-15T01:32:46.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013589599999999999 2024-09-15T01:32:46.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177021+0000", 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5268e-05 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177364+0000", 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034309 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179998+0000", 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026343040000000001 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186627+0000", 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066287209999999997 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186678+0000", 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1044000000000001e-05 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:8 3.3s0 3:c07cd22b:::smithi17529310-3:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:46.298 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.175807+0000", 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.032008984999999, 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011742891, 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 8 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175807+0000", 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175807+0000", 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175808+0000", 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5659999999999999e-06 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175816+0000", 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1770000000000001e-06 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175835+0000", 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9300000000000002e-05 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175843+0000", 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2199999999999992e-06 2024-09-15T01:32:46.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177105+0000", 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012617150000000001 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177237+0000", 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013260799999999999 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177909+0000", 2024-09-15T01:32:46.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00067179900000000005 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178042+0000", 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013286099999999999 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178152+0000", 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010978499999999999 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178189+0000", 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6859000000000001e-05 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180006+0000", 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001816757 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187405+0000", 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073992459999999999 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187550+0000", 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000144998 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:9 3.3s0 3:c07cd22b:::smithi17529310-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.175883+0000", 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.031932683000001, 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01603094, 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.305 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175883+0000", 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175883+0000", 2024-09-15T01:32:46.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175884+0000", 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175886+0000", 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.429e-06 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175893+0000", 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6630000000000004e-06 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175900+0000", 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999997e-06 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178335+0000", 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024349200000000001 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178422+0000", 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7049e-05 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187881+0000", 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0094586740000000002 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187983+0000", 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102005 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.191914+0000", 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039315039999999997 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 34/31 ECSubWrite(tid=10, reqid=client.4477.0:122, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.270782+0000", 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.9370340150000001, 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062423779999999998, 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.310 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270782+0000", 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270782+0000", 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270782+0000", 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5599999999999995e-07 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.270792+0000", 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9240000000000002e-06 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276796+0000", 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060040240000000002 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276804+0000", 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2819999999999999e-06 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276840+0000", 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5920000000000002e-05 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276846+0000", 2024-09-15T01:32:46.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0750000000000003e-06 2024-09-15T01:32:46.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276905+0000", 2024-09-15T01:32:46.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9462000000000001e-05 2024-09-15T01:32:46.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276914+0000", 2024-09-15T01:32:46.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4319999999999999e-06 2024-09-15T01:32:46.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276945+0000", 2024-09-15T01:32:46.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1334999999999997e-05 2024-09-15T01:32:46.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.276959+0000", 2024-09-15T01:32:46.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3730999999999999e-05 2024-09-15T01:32:46.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277024+0000", 2024-09-15T01:32:46.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5636999999999995e-05 2024-09-15T01:32:46.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.316 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.316 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 34/31 ECSubWrite(tid=13, reqid=client.4477.0:128, at_version=34'8, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:32:46.316 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:32:46.316 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.9305019970000004, 2024-09-15T01:32:46.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.101528777, 2024-09-15T01:32:46.316 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7999999999999995e-07 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277326+0000", 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.184e-05 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378494+0000", 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.101167374 2024-09-15T01:32:46.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378498+0000", 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1150000000000004e-06 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378565+0000", 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7354999999999995e-05 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378574+0000", 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000007e-06 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378662+0000", 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8287000000000004e-05 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378672+0000", 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3840000000000003e-06 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378705+0000", 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3506000000000001e-05 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378722+0000", 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6449000000000001e-05 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378843+0000", 2024-09-15T01:32:46.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000120937 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:154 3.3s0 3:cf3ce551:::smithi17529310-22:head [stat,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.307955+0000", 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.8998604849999996, 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066651360000000003, 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:46.322 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 154 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307955+0000", 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307955+0000", 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307957+0000", 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3009999999999999e-06 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307964+0000", 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9840000000000004e-06 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307976+0000", 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2626000000000001e-05 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307981+0000", 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4530000000000004e-06 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308022+0000", 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1016999999999997e-05 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308090+0000", 2024-09-15T01:32:46.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8688000000000004e-05 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308416+0000", 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032565200000000001 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308515+0000", 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8859999999999999e-05 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308523+0000", 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.554e-06 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308559+0000", 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6579999999999999e-05 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308575+0000", 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5515999999999998e-05 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.310704+0000", 2024-09-15T01:32:46.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021298519999999998 2024-09-15T01:32:46.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.314621+0000", 2024-09-15T01:32:46.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039160530000000001 2024-09-15T01:32:46.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.328 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 38/31 ECSubWrite(tid=51, reqid=client.4477.0:200, at_version=38'16, trim_to=0'0, roll_forward_to=34'13))", 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.206055+0000", 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0017604090000001, 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073109760000000003, 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206055+0000", 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206055+0000", 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206056+0000", 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206073+0000", 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7085999999999999e-05 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206099+0000", 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5914999999999998e-05 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206104+0000", 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7380000000000002e-06 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206140+0000", 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6007000000000003e-05 2024-09-15T01:32:46.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206147+0000", 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5710000000000003e-06 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206209+0000", 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2593000000000002e-05 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206215+0000", 2024-09-15T01:32:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5790000000000003e-06 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206236+0000", 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1199000000000002e-05 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206244+0000", 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5909999999999996e-06 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.213366+0000", 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071219940000000004 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 38/31 ECSubWrite(tid=52, reqid=client.4477.0:201, at_version=38'17, trim_to=0'0, roll_forward_to=34'13))", 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.206833+0000", 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0009827299999996, 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006658232, 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.334 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206833+0000", 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206833+0000", 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206834+0000", 2024-09-15T01:32:46.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206848+0000", 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4222999999999999e-05 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206874+0000", 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6342e-05 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206879+0000", 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206915+0000", 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5809000000000002e-05 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206920+0000", 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-06 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206981+0000", 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0729000000000001e-05 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.206989+0000", 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2320000000000001e-06 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.207023+0000", 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3818e-05 2024-09-15T01:32:46.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.207033+0000", 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.944e-06 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.213491+0000", 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006458353 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:222 3.3s0 3:c07cd22b:::smithi17529310-3:head [append 0~622592] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:41.213888+0000", 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.993928146, 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071143150000000004, 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 222 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.213888+0000", 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.213888+0000", 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.213890+0000", 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-09-15T01:32:46.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.217466+0000", 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035758719999999999 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.217490+0000", 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3866000000000002e-05 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.217496+0000", 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3620000000000002e-06 2024-09-15T01:32:46.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.217540+0000", 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4209000000000003e-05 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.217619+0000", 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9256000000000001e-05 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.218509+0000", 2024-09-15T01:32:46.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00088994599999999997 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.218860+0000", 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035033699999999999 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.218869+0000", 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2560000000000008e-06 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.219028+0000", 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015919799999999999 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.220584+0000", 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001555702 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.220926+0000", 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034254700000000001 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:46.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:46.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.221002+0000", 2024-09-15T01:32:46.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5593000000000006e-05 2024-09-15T01:32:46.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:46.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:46.346 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:46.347 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-15T01:32:46.349 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:32:46.350 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T01:32:46.350 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T01:32:46.355 INFO:tasks.rados.rados.0.smithi175.stdout:102: done (1 left) 2024-09-15T01:32:46.355 INFO:tasks.rados.rados.0.smithi175.stdout:103: delete oid 38 current snap is 5 2024-09-15T01:32:46.355 INFO:tasks.rados.rados.0.smithi175.stdout:100: finishing write tid 1 to smithi17529310-7 2024-09-15T01:32:46.355 INFO:tasks.rados.rados.0.smithi175.stdout:100: finishing write tid 2 to smithi17529310-7 2024-09-15T01:32:46.355 INFO:tasks.rados.rados.0.smithi175.stdout:100: finishing write tid 3 to smithi17529310-7 2024-09-15T01:32:46.355 INFO:tasks.rados.rados.0.smithi175.stdout:100: finishing write tid 4 to smithi17529310-7 2024-09-15T01:32:46.355 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 7 v 18 (ObjNum 70 snap 5 seq_num 70) dirty exists 2024-09-15T01:32:46.355 INFO:tasks.rados.rados.0.smithi175.stdout:100: left oid 7 (ObjNum 70 snap 5 seq_num 70) 2024-09-15T01:32:46.355 INFO:tasks.rados.rados.0.smithi175.stdout:100: done (1 left) 2024-09-15T01:32:46.355 INFO:tasks.rados.rados.0.smithi175.stdout:103: done (0 left) 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:104: delete oid 28 current snap is 5 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:104: done (0 left) 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:105: copy_from oid 39 from oid 12 current snap is 5 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:106: read oid 2 snap -1 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:106: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 43 current snap is 5 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:107: seq_num 73 ranges {147456=458752,606208=475136,1081344=294912} 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:107: writing smithi17529310-43 from 147456 to 606208 tid 1 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:107: writing smithi17529310-43 from 606208 to 1081344 tid 2 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:107: writing smithi17529310-43 from 1081344 to 1376256 tid 3 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:108: setattr oid 30 current snap is 5 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:105: finishing copy_from racing read to smithi17529310-39 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:109: rollback oid 32 current snap is 5 2024-09-15T01:32:46.356 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 32 to 4 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:107: finishing write tid 1 to smithi17529310-43 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:110: rmattr oid 1 current snap is 5 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:110: done (5 left) 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:111: read oid 18 snap -1 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:111: expect (ObjNum 17 snap 0 seq_num 17) 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:112: rmattr oid 45 current snap is 5 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:112: done (6 left) 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:113: read oid 6 snap -1 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:113: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:114: read oid 22 snap -1 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:114: expect deleted 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:115: rollback oid 31 current snap is 5 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 31 to 4 2024-09-15T01:32:46.357 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 13 current snap is 5 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:116: seq_num 74 ranges {0=475136,475136=622592,1097728=147456} 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:116: writing smithi17529310-13 from 0 to 475136 tid 1 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:116: writing smithi17529310-13 from 475136 to 1097728 tid 2 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:116: writing smithi17529310-13 from 1097728 to 1245184 tid 3 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:117: delete oid 24 current snap is 5 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:107: finishing write tid 2 to smithi17529310-43 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:107: finishing write tid 3 to smithi17529310-43 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:107: finishing write tid 4 to smithi17529310-43 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:111: expect (ObjNum 17 snap 0 seq_num 17) 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:105: finishing copy_from to smithi17529310-39 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 39 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:109: finishing rollback tid 0 to smithi17529310-32 2024-09-15T01:32:46.358 INFO:tasks.rados.rados.0.smithi175.stdout:105: done (10 left) 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:111: done (9 left) 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:114: done (8 left) 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:117: done (7 left) 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 14 current snap is 5 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 30 v 14 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:109: finishing rollback tid 1 to smithi17529310-32 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:113: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:109: finishing rollback tid 2 to smithi17529310-32 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 32 v 25 (ObjNum 63 snap 3 seq_num 63) dirty exists 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:106: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:116: finishing write tid 1 to smithi17529310-13 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:116: finishing write tid 2 to smithi17529310-13 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:116: finishing write tid 3 to smithi17529310-13 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:118: seq_num 76 ranges {1179648=507904,1687552=507904,2195456=262144} 2024-09-15T01:32:46.359 INFO:tasks.rados.rados.0.smithi175.stdout:118: writing smithi17529310-14 from 1179648 to 1687552 tid 1 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:118: writing smithi17529310-14 from 1687552 to 2195456 tid 2 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:118: writing smithi17529310-14 from 2195456 to 2457600 tid 3 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:106: done (7 left) 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:108: done (6 left) 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:109: done (5 left) 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:113: done (4 left) 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:119: delete oid 41 current snap is 5 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:115: finishing rollback tid 0 to smithi17529310-31 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:115: finishing rollback tid 1 to smithi17529310-31 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:115: finishing rollback tid 2 to smithi17529310-31 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 31 v 12 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:107: finishing write tid 5 to smithi17529310-43 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 43 v 25 (ObjNum 73 snap 5 seq_num 73) dirty exists 2024-09-15T01:32:46.360 INFO:tasks.rados.rados.0.smithi175.stdout:107: left oid 43 (ObjNum 73 snap 5 seq_num 73) 2024-09-15T01:32:46.361 INFO:tasks.rados.rados.0.smithi175.stdout:118: finishing write tid 1 to smithi17529310-14 2024-09-15T01:32:46.361 INFO:tasks.rados.rados.0.smithi175.stdout:107: done (4 left) 2024-09-15T01:32:46.361 INFO:tasks.rados.rados.0.smithi175.stdout:115: done (3 left) 2024-09-15T01:32:46.361 INFO:tasks.rados.rados.0.smithi175.stdout:119: done (2 left) 2024-09-15T01:32:46.361 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 48 current snap is 5 2024-09-15T01:32:46.361 INFO:tasks.rados.rados.0.smithi175.stdout:120: seq_num 78 ranges {2768896=557056,3325952=671744,3997696=98304} 2024-09-15T01:32:46.361 INFO:tasks.rados.rados.0.smithi175.stdout:120: writing smithi17529310-48 from 2768896 to 3325952 tid 1 2024-09-15T01:32:46.361 INFO:tasks.rados.rados.0.smithi175.stdout:120: writing smithi17529310-48 from 3325952 to 3997696 tid 2 2024-09-15T01:32:46.361 INFO:tasks.rados.rados.0.smithi175.stdout:120: writing smithi17529310-48 from 3997696 to 4096000 tid 3 2024-09-15T01:32:46.361 INFO:tasks.rados.rados.0.smithi175.stdout:121: snap_remove snap 4 2024-09-15T01:32:46.361 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:46.346+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:46.448 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:46.447+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:46.452 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:46.452 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:46.452 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:32:46.452 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:46.460 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:32:46.549 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:46.548+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:46.558 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:46.558 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:46.558 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:32:46.558 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:32:46.558 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:46.566 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:32:46.650 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:46.649+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:46.707 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:46.707 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:32:46.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:32:46.707 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:32:46.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.707 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=152, reqid=osd.13.0:152, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:46.707 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.243665+0000", 2024-09-15T01:32:46.707 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.46055407399999998, 2024-09-15T01:32:46.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019478882999999999, 2024-09-15T01:32:46.707 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.707 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.707 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243665+0000", 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243665+0000", 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243666+0000", 2024-09-15T01:32:46.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.34e-07 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243686+0000", 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9830000000000002e-05 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243778+0000", 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2367999999999994e-05 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243802+0000", 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3646e-05 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243862+0000", 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0579999999999999e-05 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243876+0000", 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4025e-05 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245801+0000", 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0019244399999999999 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246115+0000", 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00031487699999999997 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248315+0000", 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021992140000000001 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248329+0000", 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4450000000000001e-05 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263144+0000", 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014815218999999999 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:32:46.712 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.45725147199999999, 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019581815999999998, 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246969+0000", 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7000000000000003e-07 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247008+0000", 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9354999999999998e-05 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247051+0000", 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2849000000000003e-05 2024-09-15T01:32:46.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247060+0000", 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4589999999999998e-06 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248680+0000", 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001620058 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248690+0000", 2024-09-15T01:32:46.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8660000000000001e-06 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248945+0000", 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025509900000000001 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251105+0000", 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021604290000000002 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251177+0000", 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1680999999999994e-05 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251189+0000", 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2661e-05 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266550+0000", 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015360389 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.717 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=28, reqid=osd.8.0:28, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.45406960099999999, 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022199446000000001, 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4700000000000001e-07 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250167+0000", 2024-09-15T01:32:46.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7384000000000002e-05 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250202+0000", 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4912e-05 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250210+0000", 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6029999999999999e-06 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250258+0000", 2024-09-15T01:32:46.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8109999999999998e-05 2024-09-15T01:32:46.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250265+0000", 2024-09-15T01:32:46.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4400000000000002e-06 2024-09-15T01:32:46.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250466+0000", 2024-09-15T01:32:46.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000201227 2024-09-15T01:32:46.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251113+0000", 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00064769299999999997 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251179+0000", 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5895999999999994e-05 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251196+0000", 2024-09-15T01:32:46.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6810999999999999e-05 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272349+0000", 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021153222999999999 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.45271547299999998, 2024-09-15T01:32:46.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023186075, 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251505+0000", 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.09e-07 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251531+0000", 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5834999999999999e-05 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251574+0000", 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2879000000000001e-05 2024-09-15T01:32:46.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251580+0000", 2024-09-15T01:32:46.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8460000000000002e-06 2024-09-15T01:32:46.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251639+0000", 2024-09-15T01:32:46.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8331000000000001e-05 2024-09-15T01:32:46.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251649+0000", 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9930000000000007e-06 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251890+0000", 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00024173400000000001 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251923+0000", 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2607999999999997e-05 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251985+0000", 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2238999999999998e-05 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251998+0000", 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2979999999999999e-05 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274690+0000", 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022691821000000001 2024-09-15T01:32:46.728 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s1 44/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=44'26))", 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.45149847500000001, 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019696631999999999, 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252722+0000", 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3099999999999996e-07 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252734+0000", 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1576999999999999e-05 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252780+0000", 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6820999999999998e-05 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252788+0000", 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2710000000000003e-06 2024-09-15T01:32:46.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252848+0000", 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0718999999999999e-05 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252859+0000", 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0342e-05 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252960+0000", 2024-09-15T01:32:46.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000101741 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252970+0000", 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0008e-05 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253967+0000", 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00099612599999999996 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253987+0000", 2024-09-15T01:32:46.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0064e-05 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018431032 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=154, reqid=client.4477.0:282, at_version=44'19, trim_to=0'0, roll_forward_to=44'17))", 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.44748681499999998, 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023652260000000001, 2024-09-15T01:32:46.734 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256734+0000", 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3900000000000004e-07 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256754+0000", 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0279e-05 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256785+0000", 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0703000000000001e-05 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256798+0000", 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3606e-05 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256843+0000", 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5065000000000003e-05 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256851+0000", 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5580000000000003e-06 2024-09-15T01:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257013+0000", 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016264 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257339+0000", 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032564200000000001 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257402+0000", 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3162000000000003e-05 2024-09-15T01:32:46.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258083+0000", 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00068126099999999995 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280385+0000", 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022301504999999999 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=85, reqid=osd.11.0:84, at_version=0'0, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.44716303099999999, 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020078747000000001, 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257057+0000", 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5000000000000002e-07 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257067+0000", 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5540000000000003e-06 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257097+0000", 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9892000000000001e-05 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257102+0000", 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3360000000000002e-06 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257155+0000", 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3313000000000003e-05 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257164+0000", 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1060000000000006e-06 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257257+0000", 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2678999999999998e-05 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257341+0000", 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3459000000000002e-05 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257403+0000", 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2477000000000005e-05 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257420+0000", 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7170999999999999e-05 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277135+0000", 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019715010000000002 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=30, reqid=osd.8.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.44689575999999998, 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020096306000000001, 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3099999999999997e-07 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257335+0000", 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0727000000000001e-05 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257371+0000", 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5583999999999997e-05 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257378+0000", 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.754e-06 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257427+0000", 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9048999999999997e-05 2024-09-15T01:32:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257475+0000", 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8140999999999999e-05 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257605+0000", 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012981700000000001 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257640+0000", 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.557e-05 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257674+0000", 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4215999999999999e-05 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257692+0000", 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7988000000000001e-05 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277420+0000", 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019727728999999999 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=86, reqid=osd.11.0:86, at_version=44'42, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.44630450599999999, 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022807320999999998, 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257916+0000", 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257951+0000", 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5754999999999999e-05 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257982+0000", 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0732000000000003e-05 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257987+0000", 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1900000000000003e-06 2024-09-15T01:32:46.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258036+0000", 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8547999999999997e-05 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258047+0000", 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1082e-05 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258512+0000", 2024-09-15T01:32:46.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00046539599999999997 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258527+0000", 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5121e-05 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258568+0000", 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0305000000000001e-05 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258581+0000", 2024-09-15T01:32:46.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3434999999999999e-05 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280722+0000", 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022141137000000002 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=155, reqid=client.4477.0:282, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.44537970900000001, 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021774615000000001, 2024-09-15T01:32:46.755 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258841+0000", 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5899999999999995e-07 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258853+0000", 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2115e-05 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258889+0000", 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6007000000000003e-05 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258896+0000", 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258958+0000", 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2242999999999993e-05 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258966+0000", 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2370000000000003e-06 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259070+0000", 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000104171 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259104+0000", 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4569000000000001e-05 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259140+0000", 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5698000000000003e-05 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259155+0000", 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4541e-05 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280614+0000", 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021459865000000002 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.760 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.444950334, 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027513342999999999, 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259270+0000", 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.33e-07 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259295+0000", 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4413000000000002e-05 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259335+0000", 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0154999999999998e-05 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259344+0000", 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8890000000000008e-06 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259414+0000", 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9929000000000001e-05 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259423+0000", 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0289999999999994e-06 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263376+0000", 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003953036 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263393+0000", 2024-09-15T01:32:46.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7444000000000001e-05 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263481+0000", 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8251999999999995e-05 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266784+0000", 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003303074 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286783+0000", 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019998189 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=31, reqid=client.4477.0:288, at_version=44'22, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.44018296299999998, 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022720738000000001, 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.766 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:32:46.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6599999999999995e-07 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264398+0000", 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00036057299999999999 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264436+0000", 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7968e-05 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264444+0000", 2024-09-15T01:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8949999999999998e-06 2024-09-15T01:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264489+0000", 2024-09-15T01:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4954999999999999e-05 2024-09-15T01:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264499+0000", 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0574000000000001e-05 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266178+0000", 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001678543 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266188+0000", 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0624e-05 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268728+0000", 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025397240000000001 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268912+0000", 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018415400000000001 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286757+0000", 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.017844961999999999 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=32, reqid=client.4477.0:289, at_version=44'23, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.43698352499999998, 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019719520000000001, 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267237+0000", 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3099999999999998e-07 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267495+0000", 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025857700000000001 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267528+0000", 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2617999999999998e-05 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267534+0000", 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0159999999999997e-06 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267584+0000", 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0176999999999997e-05 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267594+0000", 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0161e-05 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267768+0000", 2024-09-15T01:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017420000000000001 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268732+0000", 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00096413399999999995 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268818+0000", 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5583999999999999e-05 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268943+0000", 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012452399999999999 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286956+0000", 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018012997999999999 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=91, reqid=osd.11.0:87, at_version=0'0, trim_to=0'0, roll_forward_to=44'44))", 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.43689108100000001, 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019815672999999999, 2024-09-15T01:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267329+0000", 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7500000000000001e-07 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267339+0000", 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0351e-05 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267385+0000", 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6008999999999999e-05 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267393+0000", 2024-09-15T01:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7810000000000005e-06 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267443+0000", 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0139000000000002e-05 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267454+0000", 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1012999999999999e-05 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267585+0000", 2024-09-15T01:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013078699999999999 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268732+0000", 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001146768 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268817+0000", 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5724999999999997e-05 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268838+0000", 2024-09-15T01:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0954e-05 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.287144+0000", 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018305770999999998 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=33, reqid=client.4477.0:290, at_version=44'24, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.268479+0000", 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.43574021600000001, 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018839779000000001, 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268479+0000", 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268479+0000", 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268480+0000", 2024-09-15T01:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268607+0000", 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012669499999999999 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268635+0000", 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8778000000000001e-05 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268642+0000", 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1539999999999999e-06 2024-09-15T01:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268696+0000", 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4148999999999999e-05 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268706+0000", 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9229999999999997e-06 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268871+0000", 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016481300000000001 2024-09-15T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268904+0000", 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3188999999999997e-05 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271557+0000", 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0026537380000000001 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271613+0000", 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5708999999999998e-05 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.287319+0000", 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015705909000000001 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=34, reqid=client.4477.0:291, at_version=44'25, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.269141+0000", 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.435078252, 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018283911999999999, 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269141+0000", 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269141+0000", 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269142+0000", 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6700000000000003e-07 2024-09-15T01:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269160+0000", 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7705000000000001e-05 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269207+0000", 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7574999999999998e-05 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269216+0000", 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0140000000000004e-06 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269258+0000", 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2005999999999997e-05 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269269+0000", 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.11e-05 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269344+0000", 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4220000000000004e-05 2024-09-15T01:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271566+0000", 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022224100000000002 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271605+0000", 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8878000000000003e-05 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271643+0000", 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8068000000000002e-05 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.287425+0000", 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015782168999999999 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=32, reqid=client.4477.0:294, at_version=44'23, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.43391686800000001, 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021391184000000001, 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270304+0000", 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7499999999999998e-07 2024-09-15T01:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270326+0000", 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2821999999999999e-05 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270366+0000", 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9708e-05 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270374+0000", 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6709999999999999e-06 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270433+0000", 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9274999999999997e-05 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270444+0000", 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.131e-05 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271974+0000", 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001529996 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271986+0000", 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1443e-05 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272180+0000", 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019405299999999999 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272366+0000", 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018627699999999999 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.291694+0000", 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019327654 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.43332018999999999, 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024535670999999998, 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270900+0000", 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.23e-07 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270920+0000", 2024-09-15T01:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.022e-05 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270953+0000", 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3136000000000003e-05 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270960+0000", 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4280000000000001e-06 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272041+0000", 2024-09-15T01:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010816440000000001 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272050+0000", 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5269999999999998e-06 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272149+0000", 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9464999999999997e-05 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272182+0000", 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2792e-05 2024-09-15T01:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272209+0000", 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6409000000000001e-05 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272435+0000", 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000226691 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295435+0000", 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022999636 2024-09-15T01:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=34, reqid=client.4477.0:296, at_version=44'25, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.43275408300000001, 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024251137999999998, 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271466+0000", 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8899999999999998e-07 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271488+0000", 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1875999999999999e-05 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271522+0000", 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4041999999999997e-05 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271529+0000", 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4620000000000001e-06 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272189+0000", 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066068699999999997 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272198+0000", 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2900000000000002e-06 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272284+0000", 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6638999999999995e-05 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272322+0000", 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7125000000000001e-05 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272357+0000", 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5626000000000001e-05 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272458+0000", 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000100343 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295717+0000", 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023259058999999999 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=35, reqid=client.4477.0:291, at_version=0'0, trim_to=0'0, roll_forward_to=44'25))", 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.43114694599999998, 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022393413000000001, 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0999999999999998e-07 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273083+0000", 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0015e-05 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273110+0000", 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6798999999999999e-05 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273117+0000", 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5379999999999999e-06 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273155+0000", 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8563999999999999e-05 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273163+0000", 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4939999999999997e-06 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273248+0000", 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5290000000000005e-05 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273254+0000", 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3149999999999997e-06 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273288+0000", 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3958000000000003e-05 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273303+0000", 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4731e-05 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295466+0000", 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022162998999999999 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:46.812 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:32:46.813 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:46.811+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:46.814 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:32:46.912 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:46.912 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:46.912 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:32:46.912 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:46.913 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:46.911+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:46.920 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:32:46.940 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:32:46.940 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:32:46.959 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:32:46.959 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:32:46.980 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:32:46.980 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:32:46.999 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:32:46.999 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:32:47.014 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:47.012+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:47.017 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:32:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:32:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:32:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:32:47.017 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:32:47.017 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:32:47.018 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:32:47.025 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:32:47.037 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:32:47.037 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:32:47.056 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:32:47.057 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:32:47.076 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:32:47.076 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:32:47.098 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:32:47.099 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:32:47.115 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:47.113+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:47.119 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:32:47.119 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:32:47.121 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:47.121 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:47.130 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:32:47.138 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:32:47.138 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:32:47.158 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:32:47.158 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:32:47.177 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:32:47.177 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:32:47.196 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:32:47.196 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:32:47.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:32:47.195+0000 7f48d5664640 0 -- 172.21.15.42:0/2074842977 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7f48d0192aa0 msgr2=0x7f48d0194e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-15T01:32:47.216 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:47.215+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:47.226 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:47.227 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:47.235 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:32:47.239 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T01:32:47.241 INFO:tasks.rados.rados.0.smithi175.stdout:121: done (3 left) 2024-09-15T01:32:47.242 INFO:tasks.rados.rados.0.smithi175.stdout:122: rmattr oid 25 current snap is 5 2024-09-15T01:32:47.243 INFO:tasks.rados.rados.0.smithi175.stdout:122: done (3 left) 2024-09-15T01:32:47.243 INFO:tasks.rados.rados.0.smithi175.stdout:123: rollback oid 32 current snap is 5 2024-09-15T01:32:47.243 INFO:tasks.rados.rados.0.smithi175.stdout:123: done (3 left) 2024-09-15T01:32:47.243 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 32 current snap is 5 2024-09-15T01:32:47.243 INFO:tasks.rados.rados.0.smithi175.stdout:124: seq_num 79 ranges {2555904=573440,3129344=425984,3555328=344064} 2024-09-15T01:32:47.257 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T01:32:47.260 INFO:tasks.rados.rados.0.smithi175.stdout:124: writing smithi17529310-32 from 2555904 to 3129344 tid 1 2024-09-15T01:32:47.263 INFO:tasks.rados.rados.0.smithi175.stdout:124: writing smithi17529310-32 from 3129344 to 3555328 tid 2 2024-09-15T01:32:47.265 INFO:tasks.rados.rados.0.smithi175.stdout:124: writing smithi17529310-32 from 3555328 to 3899392 tid 3 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:125: copy_from oid 17 from oid 19 current snap is 5 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:118: finishing write tid 2 to smithi17529310-14 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:118: finishing write tid 3 to smithi17529310-14 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:118: finishing write tid 4 to smithi17529310-14 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:118: finishing write tid 5 to smithi17529310-14 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 14 v 23 (ObjNum 76 snap 5 seq_num 76) dirty exists 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:118: left oid 14 (ObjNum 76 snap 5 seq_num 76) 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:116: finishing write tid 4 to smithi17529310-13 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:116: finishing write tid 5 to smithi17529310-13 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 13 v 34 (ObjNum 74 snap 5 seq_num 74) dirty exists 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:116: left oid 13 (ObjNum 74 snap 5 seq_num 74) 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:120: finishing write tid 1 to smithi17529310-48 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:120: finishing write tid 2 to smithi17529310-48 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:120: finishing write tid 3 to smithi17529310-48 2024-09-15T01:32:47.266 INFO:tasks.rados.rados.0.smithi175.stdout:116: done (4 left) 2024-09-15T01:32:47.267 INFO:tasks.rados.rados.0.smithi175.stdout:118: done (3 left) 2024-09-15T01:32:47.267 INFO:tasks.rados.rados.0.smithi175.stdout:126: rmattr oid 34 current snap is 5 2024-09-15T01:32:47.267 INFO:tasks.rados.rados.0.smithi175.stdout:126: done (3 left) 2024-09-15T01:32:47.267 INFO:tasks.rados.rados.0.smithi175.stdout:127: rmattr oid 47 current snap is 5 2024-09-15T01:32:47.267 INFO:tasks.rados.rados.0.smithi175.stdout:127: done (3 left) 2024-09-15T01:32:47.267 INFO:tasks.rados.rados.0.smithi175.stdout:128: snap_create 2024-09-15T01:32:47.267 INFO:tasks.rados.rados.0.smithi175.stdout:120: finishing write tid 4 to smithi17529310-48 2024-09-15T01:32:47.267 INFO:tasks.rados.rados.0.smithi175.stdout:120: finishing write tid 5 to smithi17529310-48 2024-09-15T01:32:47.267 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 48 v 39 (ObjNum 78 snap 5 seq_num 78) dirty exists 2024-09-15T01:32:47.267 INFO:tasks.rados.rados.0.smithi175.stdout:120: left oid 48 (ObjNum 78 snap 5 seq_num 78) 2024-09-15T01:32:47.267 INFO:tasks.rados.rados.0.smithi175.stdout:124: finishing write tid 1 to smithi17529310-32 2024-09-15T01:32:47.268 INFO:tasks.rados.rados.0.smithi175.stdout:124: finishing write tid 2 to smithi17529310-32 2024-09-15T01:32:47.268 INFO:tasks.rados.rados.0.smithi175.stdout:124: finishing write tid 3 to smithi17529310-32 2024-09-15T01:32:47.274 INFO:tasks.rados.rados.0.smithi175.stdout:125: finishing copy_from racing read to smithi17529310-17 2024-09-15T01:32:47.275 INFO:tasks.rados.rados.0.smithi175.stdout:125: finishing copy_from to smithi17529310-17 2024-09-15T01:32:47.275 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 17 v 4 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-15T01:32:47.317 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:47.315+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:47.338 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:47.338 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:47.346 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:32:47.416 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:32:47.416 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:32:47.417 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:47.416+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:47.436 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:32:47.436 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:32:47.484 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:47.484 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:47.492 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-15T01:32:47.519 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:47.517+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:47.589 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:47.665 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:47.667 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-15T01:32:47.667 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:47.665+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:47.764 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:47.764 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:47.767 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:47.766+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:47.772 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-15T01:32:47.868 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:47.868 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:47.869 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:47.867+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:47.876 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:32:47.880 INFO:tasks.rados.rados.0.smithi175.stdout:124: finishing write tid 4 to smithi17529310-32 2024-09-15T01:32:47.881 INFO:tasks.rados.rados.0.smithi175.stdout:124: finishing write tid 5 to smithi17529310-32 2024-09-15T01:32:47.881 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 32 v 31 (ObjNum 79 snap 5 seq_num 79) dirty exists 2024-09-15T01:32:47.881 INFO:tasks.rados.rados.0.smithi175.stdout:124: left oid 32 (ObjNum 79 snap 5 seq_num 79) 2024-09-15T01:32:47.970 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:47.968+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:47.986 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:47.986 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:47.995 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:32:48.079 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:48.078+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:48.133 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:48.134 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:48.142 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:32:48.171 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:48.169+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:48.244 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T01:32:48.245 INFO:tasks.rados.rados.0.smithi175.stdout:120: done (3 left) 2024-09-15T01:32:48.245 INFO:tasks.rados.rados.0.smithi175.stdout:124: done (2 left) 2024-09-15T01:32:48.245 INFO:tasks.rados.rados.0.smithi175.stdout:125: done (1 left) 2024-09-15T01:32:48.245 INFO:tasks.rados.rados.0.smithi175.stdout:128: done (0 left) 2024-09-15T01:32:48.245 INFO:tasks.rados.rados.0.smithi175.stdout:129: rollback oid 47 current snap is 6 2024-09-15T01:32:48.245 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 47 to 5 2024-09-15T01:32:48.245 INFO:tasks.rados.rados.0.smithi175.stdout:130: snap_create 2024-09-15T01:32:48.245 INFO:tasks.rados.rados.0.smithi175.stdout:129: finishing rollback tid 1 to smithi17529310-47 2024-09-15T01:32:48.245 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 47 v 0 (ObjNum 129 snap 0 seq_num 1041697771) dirty dne 2024-09-15T01:32:48.260 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-15T01:32:48.271 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:48.270+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:48.281 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:48.289 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:32:48.352 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:32:48.373 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:48.371+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:48.426 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:48.426 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:48.435 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:32:48.473 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:48.472+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:48.531 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:48.531 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:48.540 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:32:48.584 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:48.582+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:48.636 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:48.637 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:48.645 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:32:48.674 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:48.673+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:48.775 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:48.774+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:48.780 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:48.781 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:48.789 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:32:48.876 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:48.875+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:48.884 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:48.885 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:48.893 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:32:48.977 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:48.976+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:48.997 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-15T01:32:49.078 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:49.076+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:49.135 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:49.135 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:49.144 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-15T01:32:49.179 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:49.178+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:49.239 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:49.239 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:49.248 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-15T01:32:49.254 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-15T01:32:49.254 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-15T01:32:49.255 INFO:tasks.rados.rados.0.smithi175.stdout:129: done (1 left) 2024-09-15T01:32:49.255 INFO:tasks.rados.rados.0.smithi175.stdout:130: done (0 left) 2024-09-15T01:32:49.255 INFO:tasks.rados.rados.0.smithi175.stdout:131: rmattr oid 9 current snap is 7 2024-09-15T01:32:49.255 INFO:tasks.rados.rados.0.smithi175.stdout:131: done (0 left) 2024-09-15T01:32:49.255 INFO:tasks.rados.rados.0.smithi175.stdout:132: delete oid 17 current snap is 7 2024-09-15T01:32:49.259 INFO:tasks.rados.rados.0.smithi175.stdout:132: done (0 left) 2024-09-15T01:32:49.259 INFO:tasks.rados.rados.0.smithi175.stdout:133: rollback oid 17 current snap is 7 2024-09-15T01:32:49.259 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 17 to 5 2024-09-15T01:32:49.259 INFO:tasks.rados.rados.0.smithi175.stdout:134: rollback oid 39 current snap is 7 2024-09-15T01:32:49.259 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 39 to 5 2024-09-15T01:32:49.259 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 9 current snap is 7 2024-09-15T01:32:49.259 INFO:tasks.rados.rados.0.smithi175.stdout:135: seq_num 81 ranges {147456=606208,753664=720896,1474560=49152} 2024-09-15T01:32:49.263 INFO:tasks.rados.rados.0.smithi175.stdout:135: writing smithi17529310-9 from 147456 to 753664 tid 1 2024-09-15T01:32:49.268 INFO:tasks.rados.rados.0.smithi175.stdout:135: writing smithi17529310-9 from 753664 to 1474560 tid 2 2024-09-15T01:32:49.268 INFO:tasks.rados.rados.0.smithi175.stdout:135: writing smithi17529310-9 from 1474560 to 1523712 tid 3 2024-09-15T01:32:49.269 INFO:tasks.rados.rados.0.smithi175.stdout:136: delete oid 22 current snap is 7 2024-09-15T01:32:49.269 INFO:tasks.rados.rados.0.smithi175.stdout:133: finishing rollback tid 1 to smithi17529310-17 2024-09-15T01:32:49.269 INFO:tasks.rados.rados.0.smithi175.stdout:134: finishing rollback tid 0 to smithi17529310-39 2024-09-15T01:32:49.269 INFO:tasks.rados.rados.0.smithi175.stdout:134: finishing rollback tid 1 to smithi17529310-39 2024-09-15T01:32:49.270 INFO:tasks.rados.rados.0.smithi175.stdout:136: done (3 left) 2024-09-15T01:32:49.270 INFO:tasks.rados.rados.0.smithi175.stdout:137: rollback oid 32 current snap is 7 2024-09-15T01:32:49.270 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 32 to 5 2024-09-15T01:32:49.270 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 4 current snap is 7 2024-09-15T01:32:49.270 INFO:tasks.rados.rados.0.smithi175.stdout:138: seq_num 83 ranges {65536=638976,704512=622592,1327104=147456} 2024-09-15T01:32:49.274 INFO:tasks.rados.rados.0.smithi175.stdout:138: writing smithi17529310-4 from 65536 to 704512 tid 1 2024-09-15T01:32:49.278 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-15T01:32:49.278 INFO:tasks.rados.rados.0.smithi175.stdout:138: writing smithi17529310-4 from 704512 to 1327104 tid 2 2024-09-15T01:32:49.279 INFO:tasks.rados.rados.0.smithi175.stdout:138: writing smithi17529310-4 from 1327104 to 1474560 tid 3 2024-09-15T01:32:49.279 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 48 current snap is 7 2024-09-15T01:32:49.279 INFO:tasks.rados.rados.0.smithi175.stdout:139: seq_num 84 ranges {4096000=655360,4751360=638976,5390336=131072} 2024-09-15T01:32:49.280 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:49.278+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:49.305 INFO:tasks.rados.rados.0.smithi175.stdout:139: writing smithi17529310-48 from 4096000 to 4751360 tid 1 2024-09-15T01:32:49.309 INFO:tasks.rados.rados.0.smithi175.stdout:139: writing smithi17529310-48 from 4751360 to 5390336 tid 2 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:139: writing smithi17529310-48 from 5390336 to 5521408 tid 3 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:140: copy_from oid 10 from oid 40 current snap is 7 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:135: finishing write tid 1 to smithi17529310-9 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:135: finishing write tid 2 to smithi17529310-9 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:135: finishing write tid 3 to smithi17529310-9 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:135: finishing write tid 4 to smithi17529310-9 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:137: finishing rollback tid 0 to smithi17529310-32 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:137: finishing rollback tid 1 to smithi17529310-32 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:137: finishing rollback tid 2 to smithi17529310-32 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 32 v 35 (ObjNum 79 snap 5 seq_num 79) dirty exists 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:135: finishing write tid 5 to smithi17529310-9 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 9 v 30 (ObjNum 81 snap 7 seq_num 81) dirty exists 2024-09-15T01:32:49.311 INFO:tasks.rados.rados.0.smithi175.stdout:135: left oid 9 (ObjNum 81 snap 7 seq_num 81) 2024-09-15T01:32:49.312 INFO:tasks.rados.rados.0.smithi175.stdout:138: finishing write tid 1 to smithi17529310-4 2024-09-15T01:32:49.312 INFO:tasks.rados.rados.0.smithi175.stdout:138: finishing write tid 2 to smithi17529310-4 2024-09-15T01:32:49.312 INFO:tasks.rados.rados.0.smithi175.stdout:138: finishing write tid 3 to smithi17529310-4 2024-09-15T01:32:49.312 INFO:tasks.rados.rados.0.smithi175.stdout:135: done (6 left) 2024-09-15T01:32:49.312 INFO:tasks.rados.rados.0.smithi175.stdout:137: done (5 left) 2024-09-15T01:32:49.312 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 31 current snap is 7 2024-09-15T01:32:49.312 INFO:tasks.rados.rados.0.smithi175.stdout:138: finishing write tid 4 to smithi17529310-4 2024-09-15T01:32:49.312 INFO:tasks.rados.rados.0.smithi175.stdout:141: seq_num 85 ranges {507904=671744,1179648=770048} 2024-09-15T01:32:49.317 INFO:tasks.rados.rados.0.smithi175.stdout:141: writing smithi17529310-31 from 507904 to 1179648 tid 1 2024-09-15T01:32:49.323 INFO:tasks.rados.rados.0.smithi175.stdout:141: writing smithi17529310-31 from 1179648 to 1949696 tid 2 2024-09-15T01:32:49.323 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 32 current snap is 7 2024-09-15T01:32:49.323 INFO:tasks.rados.rados.0.smithi175.stdout:138: finishing write tid 5 to smithi17529310-4 2024-09-15T01:32:49.323 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 4 v 15 (ObjNum 83 snap 7 seq_num 83) dirty exists 2024-09-15T01:32:49.323 INFO:tasks.rados.rados.0.smithi175.stdout:138: left oid 4 (ObjNum 83 snap 7 seq_num 83) 2024-09-15T01:32:49.323 INFO:tasks.rados.rados.0.smithi175.stdout:139: finishing write tid 1 to smithi17529310-48 2024-09-15T01:32:49.323 INFO:tasks.rados.rados.0.smithi175.stdout:139: finishing write tid 2 to smithi17529310-48 2024-09-15T01:32:49.323 INFO:tasks.rados.rados.0.smithi175.stdout:139: finishing write tid 3 to smithi17529310-48 2024-09-15T01:32:49.323 INFO:tasks.rados.rados.0.smithi175.stdout:139: finishing write tid 4 to smithi17529310-48 2024-09-15T01:32:49.323 INFO:tasks.rados.rados.0.smithi175.stdout:139: finishing write tid 5 to smithi17529310-48 2024-09-15T01:32:49.323 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 48 v 46 (ObjNum 84 snap 7 seq_num 84) dirty exists 2024-09-15T01:32:49.323 INFO:tasks.rados.rados.0.smithi175.stdout:139: left oid 48 (ObjNum 84 snap 7 seq_num 84) 2024-09-15T01:32:49.324 INFO:tasks.rados.rados.0.smithi175.stdout:141: finishing write tid 1 to smithi17529310-31 2024-09-15T01:32:49.324 INFO:tasks.rados.rados.0.smithi175.stdout:142: seq_num 86 ranges {3899392=688128,4587520=540672,5128192=229376} 2024-09-15T01:32:49.347 INFO:tasks.rados.rados.0.smithi175.stdout:142: writing smithi17529310-32 from 3899392 to 4587520 tid 1 2024-09-15T01:32:49.350 INFO:tasks.rados.rados.0.smithi175.stdout:142: writing smithi17529310-32 from 4587520 to 5128192 tid 2 2024-09-15T01:32:49.351 INFO:tasks.rados.rados.0.smithi175.stdout:142: writing smithi17529310-32 from 5128192 to 5357568 tid 3 2024-09-15T01:32:49.352 INFO:tasks.rados.rados.0.smithi175.stdout:138: done (6 left) 2024-09-15T01:32:49.352 INFO:tasks.rados.rados.0.smithi175.stdout:139: done (5 left) 2024-09-15T01:32:49.352 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 19 current snap is 7 2024-09-15T01:32:49.352 INFO:tasks.rados.rados.0.smithi175.stdout:140: finishing copy_from racing read to smithi17529310-10 2024-09-15T01:32:49.352 INFO:tasks.rados.rados.0.smithi175.stdout:143: seq_num 87 ranges {311296=704512,1015808=688128,1703936=81920} 2024-09-15T01:32:49.356 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:49.356 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:49.357 INFO:tasks.rados.rados.0.smithi175.stdout:143: writing smithi17529310-19 from 311296 to 1015808 tid 1 2024-09-15T01:32:49.361 INFO:tasks.rados.rados.0.smithi175.stdout:143: writing smithi17529310-19 from 1015808 to 1703936 tid 2 2024-09-15T01:32:49.362 INFO:tasks.rados.rados.0.smithi175.stdout:143: writing smithi17529310-19 from 1703936 to 1785856 tid 3 2024-09-15T01:32:49.362 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 45 current snap is 7 2024-09-15T01:32:49.362 INFO:tasks.rados.rados.0.smithi175.stdout:140: finishing copy_from to smithi17529310-10 2024-09-15T01:32:49.362 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 10 v 10 (ObjNum 58 snap 3 seq_num 58) dirty exists 2024-09-15T01:32:49.362 INFO:tasks.rados.rados.0.smithi175.stdout:141: finishing write tid 2 to smithi17529310-31 2024-09-15T01:32:49.362 INFO:tasks.rados.rados.0.smithi175.stdout:144: seq_num 88 ranges {753664=720896,1474560=442368,1916928=327680} 2024-09-15T01:32:49.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:32:49.369 INFO:tasks.rados.rados.0.smithi175.stdout:144: writing smithi17529310-45 from 753664 to 1474560 tid 1 2024-09-15T01:32:49.372 INFO:tasks.rados.rados.0.smithi175.stdout:144: writing smithi17529310-45 from 1474560 to 1916928 tid 2 2024-09-15T01:32:49.374 INFO:tasks.rados.rados.0.smithi175.stdout:144: writing smithi17529310-45 from 1916928 to 2244608 tid 3 2024-09-15T01:32:49.374 INFO:tasks.rados.rados.0.smithi175.stdout:140: done (6 left) 2024-09-15T01:32:49.375 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 30 current snap is 7 2024-09-15T01:32:49.375 INFO:tasks.rados.rados.0.smithi175.stdout:142: finishing write tid 1 to smithi17529310-32 2024-09-15T01:32:49.375 INFO:tasks.rados.rados.0.smithi175.stdout:142: finishing write tid 2 to smithi17529310-32 2024-09-15T01:32:49.375 INFO:tasks.rados.rados.0.smithi175.stdout:141: finishing write tid 3 to smithi17529310-31 2024-09-15T01:32:49.375 INFO:tasks.rados.rados.0.smithi175.stdout:142: finishing write tid 3 to smithi17529310-32 2024-09-15T01:32:49.375 INFO:tasks.rados.rados.0.smithi175.stdout:145: seq_num 89 ranges {491520=737280,1228800=573440,1802240=196608} 2024-09-15T01:32:49.380 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:49.379+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:49.381 INFO:tasks.rados.rados.0.smithi175.stdout:145: writing smithi17529310-30 from 491520 to 1228800 tid 1 2024-09-15T01:32:49.384 INFO:tasks.rados.rados.0.smithi175.stdout:145: writing smithi17529310-30 from 1228800 to 1802240 tid 2 2024-09-15T01:32:49.386 INFO:tasks.rados.rados.0.smithi175.stdout:145: writing smithi17529310-30 from 1802240 to 1998848 tid 3 2024-09-15T01:32:49.386 INFO:tasks.rados.rados.0.smithi175.stdout:146: read oid 4 snap -1 2024-09-15T01:32:49.386 INFO:tasks.rados.rados.0.smithi175.stdout:146: expect (ObjNum 83 snap 7 seq_num 83) 2024-09-15T01:32:49.386 INFO:tasks.rados.rados.0.smithi175.stdout:147: snap_create 2024-09-15T01:32:49.386 INFO:tasks.rados.rados.0.smithi175.stdout:143: finishing write tid 1 to smithi17529310-19 2024-09-15T01:32:49.386 INFO:tasks.rados.rados.0.smithi175.stdout:143: finishing write tid 2 to smithi17529310-19 2024-09-15T01:32:49.386 INFO:tasks.rados.rados.0.smithi175.stdout:143: finishing write tid 3 to smithi17529310-19 2024-09-15T01:32:49.386 INFO:tasks.rados.rados.0.smithi175.stdout:141: finishing write tid 4 to smithi17529310-31 2024-09-15T01:32:49.386 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 31 v 20 (ObjNum 85 snap 7 seq_num 85) dirty exists 2024-09-15T01:32:49.386 INFO:tasks.rados.rados.0.smithi175.stdout:141: left oid 31 (ObjNum 85 snap 7 seq_num 85) 2024-09-15T01:32:49.386 INFO:tasks.rados.rados.0.smithi175.stdout:144: finishing write tid 1 to smithi17529310-45 2024-09-15T01:32:49.387 INFO:tasks.rados.rados.0.smithi175.stdout:144: finishing write tid 2 to smithi17529310-45 2024-09-15T01:32:49.387 INFO:tasks.rados.rados.0.smithi175.stdout:144: finishing write tid 3 to smithi17529310-45 2024-09-15T01:32:49.389 INFO:tasks.rados.rados.0.smithi175.stdout:145: finishing write tid 1 to smithi17529310-30 2024-09-15T01:32:49.389 INFO:tasks.rados.rados.0.smithi175.stdout:145: finishing write tid 2 to smithi17529310-30 2024-09-15T01:32:49.389 INFO:tasks.rados.rados.0.smithi175.stdout:145: finishing write tid 3 to smithi17529310-30 2024-09-15T01:32:49.391 INFO:tasks.rados.rados.0.smithi175.stdout:146: expect (ObjNum 83 snap 7 seq_num 83) 2024-09-15T01:32:49.448 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:32:49.473 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:49.473 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:49.481 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:49.480+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:49.482 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:32:49.508 INFO:tasks.rados.rados.0.smithi175.stdout:134: finishing rollback tid 2 to smithi17529310-39 2024-09-15T01:32:49.508 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 39 v 38 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-15T01:32:49.577 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:32:49.577 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:32:49.581 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:49.580+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:49.597 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:32:49.598 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:32:49.617 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:32:49.617 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:32:49.628 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:49.628 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:49.628 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:32:49.628 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:32:49.628 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:49.636 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:32:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:32:49.637 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:32:49.655 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:32:49.655 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:32:49.674 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:32:49.674 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:32:49.683 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:49.681+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:49.693 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:32:49.693 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:32:49.712 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:32:49.712 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:32:49.731 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:32:49.731 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:32:49.741 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:49.741 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:32:49.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:32:49.741 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:32:49.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.741 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:18 3.6s0 3:6ba0b44a:::smithi17529310-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:49.741 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.181034+0000", 2024-09-15T01:32:49.741 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.556798893, 2024-09-15T01:32:49.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085778220000000006, 2024-09-15T01:32:49.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181034+0000", 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181034+0000", 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181034+0000", 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181038+0000", 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9140000000000001e-06 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181047+0000", 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6680000000000007e-06 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181055+0000", 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0709999999999995e-06 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.183085+0000", 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020298629999999998 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.183392+0000", 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030756300000000001 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187779+0000", 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043861569999999999 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187856+0000", 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7914e-05 2024-09-15T01:32:49.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.189611+0000", 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754916 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.7s3 34/31 ECSubWrite(tid=17, reqid=client.4477.0:77, at_version=34'6, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.522328633000001, 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015250150000000001, 2024-09-15T01:32:49.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215518+0000", 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3567000000000001e-05 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230278+0000", 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014760300000000001 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230285+0000", 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.285e-06 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230359+0000", 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4025999999999999e-05 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230367+0000", 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4500000000000004e-06 2024-09-15T01:32:49.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230457+0000", 2024-09-15T01:32:49.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0457999999999993e-05 2024-09-15T01:32:49.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230498+0000", 2024-09-15T01:32:49.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0204000000000003e-05 2024-09-15T01:32:49.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230554+0000", 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6097999999999997e-05 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230573+0000", 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9252000000000001e-05 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230754+0000", 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018093300000000001 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:90 3.6s0 3:65d6c4b7:::smithi17529310-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.511690715, 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.964412163, 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 90 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226144+0000", 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0490000000000002e-06 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226152+0000", 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5700000000000004e-06 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226157+0000", 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0069999999999997e-06 2024-09-15T01:32:49.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.227618+0000", 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014613 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.227698+0000", 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9518000000000007e-05 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.228785+0000", 2024-09-15T01:32:49.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010872729999999999 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.228833+0000", 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.85e-05 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:37.179967+0000", 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.95113379099999995 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:37.179971+0000", 2024-09-15T01:32:49.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7620000000000001e-06 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:38.180851+0000", 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0008801570000001 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:38.180855+0000", 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.416e-06 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.187397+0000", 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.006542346 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.187673+0000", 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000275825 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.190554+0000", 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028811650000000002 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.7s3 34/31 ECSubRead(tid=22, to_read={3:e6e8875f:::smithi17529310-31:head=0,4096,0}, subchunks={3:e6e8875f:::smithi17529310-31:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.50431867, 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.074844447999999994, 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4300000000000003e-07 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233521+0000", 2024-09-15T01:32:49.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0280000000000003e-06 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308171+0000", 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.074650066000000001 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308178+0000", 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5810000000000002e-06 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308258+0000", 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9977999999999999e-05 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308358+0000", 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100152 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:196 3.6s0 3:65d6c4b7:::smithi17529310-30:head [append 0~0] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:32:49.762 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.5346702329999999, 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007181882, 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 196 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203168+0000", 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5620000000000003e-06 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203175+0000", 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5599999999999996e-06 2024-09-15T01:32:49.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203194+0000", 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8757999999999999e-05 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203201+0000", 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7429999999999996e-06 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203414+0000", 2024-09-15T01:32:49.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021300600000000001 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203419+0000", 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1420000000000001e-06 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.204700+0000", 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.001280647 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.204945+0000", 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024550199999999998 2024-09-15T01:32:49.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.205894+0000", 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094873799999999997 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206188+0000", 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029375700000000001 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206257+0000", 2024-09-15T01:32:49.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9632e-05 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206371+0000", 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011411100000000001 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206444+0000", 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2355999999999996e-05 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210281+0000", 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038376740000000001 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210344+0000", 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2694000000000007e-05 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:197 3.6s0 3:65d6c4b7:::smithi17529310-30:head [rollback 3] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.5346166389999993, 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0090962939999999, 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 197 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8299999999999997e-07 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203220+0000", 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0620000000000002e-06 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203229+0000", 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5329999999999994e-06 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203234+0000", 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000002e-06 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203537+0000", 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030286700000000001 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203539+0000", 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.277e-06 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206748+0000", 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.003209102 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206862+0000", 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011395000000000001 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207832+0000", 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097048299999999996 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207937+0000", 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104583 2024-09-15T01:32:49.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207956+0000", 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9381e-05 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208001+0000", 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4642999999999999e-05 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209901+0000", 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001899766 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211474+0000", 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015733190000000001 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212312+0000", 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083790500000000001 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.776 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:198 3.6s0 3:65d6c4b7:::smithi17529310-30:head [append 0~0] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.5343324470000006, 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.009021902, 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 198 2024-09-15T01:32:49.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203501+0000", 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4600000000000004e-07 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203505+0000", 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3830000000000002e-06 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203562+0000", 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7120000000000002e-05 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203570+0000", 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203656+0000", 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6273000000000006e-05 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203658+0000", 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4360000000000001e-06 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208296+0000", 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004637574 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208407+0000", 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110994 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209048+0000", 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064081900000000005 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209141+0000", 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3827999999999997e-05 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209164+0000", 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2201e-05 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209206+0000", 2024-09-15T01:32:49.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2021999999999998e-05 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209910+0000", 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070434399999999998 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211939+0000", 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020290159999999998 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212522+0000", 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058283599999999999 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:211 3.6s0 3:6ba0b44a:::smithi17529310-6:head [copy-from ver 0] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.5398519200000003, 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.22039597699999999, 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:49.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:32:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197982+0000", 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.618e-06 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197988+0000", 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4519999999999996e-06 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198027+0000", 2024-09-15T01:32:49.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9492000000000001e-05 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198036+0000", 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3950000000000007e-06 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198736+0000", 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00069949200000000002 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198740+0000", 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3470000000000001e-06 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210425+0000", 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011684905000000001 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210622+0000", 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019688099999999999 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418376+0000", 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.20775439500000001 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:212 3.6s0 3:6ba0b44a:::smithi17529310-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.5397882710000008, 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.22052332099999999, 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:49.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 212 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198045+0000", 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6499999999999998e-07 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198049+0000", 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4150000000000001e-06 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198058+0000", 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9830000000000002e-06 2024-09-15T01:32:49.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198064+0000", 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6219999999999997e-06 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198782+0000", 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071748199999999997 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198783+0000", 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9400000000000001e-06 2024-09-15T01:32:49.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210808+0000", 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012024217 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210861+0000", 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2859999999999999e-05 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418443+0000", 2024-09-15T01:32:49.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.20758253099999999 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418509+0000", 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5516000000000001e-05 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418567+0000", 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.889e-05 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.794 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs6 39/31 ECSubWrite(tid=56, reqid=osd.11.0:56, at_version=39'21, trim_to=0'0, roll_forward_to=38'17))", 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4950395190000005, 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038235071000000002, 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242794+0000", 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8599999999999997e-07 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242817+0000", 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3655e-05 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280383+0000", 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037566100999999998 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280390+0000", 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280442+0000", 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1650000000000002e-05 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280448+0000", 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7479999999999999e-06 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280660+0000", 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021205900000000001 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280667+0000", 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4710000000000001e-06 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280705+0000", 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7511000000000001e-05 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280722+0000", 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7646999999999999e-05 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281028+0000", 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030575099999999999 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s1 44/31 ECSubWrite(tid=29, reqid=osd.5.0:29, at_version=44'21, trim_to=0'0, roll_forward_to=40'19))", 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.240231+0000", 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4976016219999999, 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0092854300000000008, 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240231+0000", 2024-09-15T01:32:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240231+0000", 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240231+0000", 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3200000000000001e-07 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240255+0000", 2024-09-15T01:32:49.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3978999999999999e-05 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240354+0000", 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8795000000000005e-05 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240361+0000", 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7109999999999998e-06 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240412+0000", 2024-09-15T01:32:49.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1104999999999999e-05 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240422+0000", 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0603e-05 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241395+0000", 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097287000000000003 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241408+0000", 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.273e-05 2024-09-15T01:32:49.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241441+0000", 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3243e-05 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241457+0000", 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6324000000000002e-05 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249516+0000", 2024-09-15T01:32:49.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080588380000000005 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs2 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4955174470000001, 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011429094000000001, 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242316+0000", 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2900000000000003e-07 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242337+0000", 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1263999999999998e-05 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242392+0000", 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5408999999999998e-05 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242399+0000", 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.866e-06 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242456+0000", 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-05 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242466+0000", 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0015e-05 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242685+0000", 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021821399999999999 2024-09-15T01:32:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242713+0000", 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8589e-05 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242754+0000", 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1173e-05 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242790+0000", 2024-09-15T01:32:49.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5167999999999999e-05 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253744+0000", 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010954527 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs1 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4950210629999998, 2024-09-15T01:32:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010913366000000001, 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242812+0000", 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242834+0000", 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1827999999999998e-05 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242901+0000", 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6840000000000004e-05 2024-09-15T01:32:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242908+0000", 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5440000000000001e-06 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242959+0000", 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1057000000000003e-05 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242969+0000", 2024-09-15T01:32:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8030000000000008e-06 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243189+0000", 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021987499999999999 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243212+0000", 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2569999999999999e-05 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243258+0000", 2024-09-15T01:32:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6452000000000002e-05 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243274+0000", 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6426000000000002e-05 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253725+0000", 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010450249 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs6 44/31 ECSubWrite(tid=82, reqid=client.4477.0:281, at_version=44'36, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4942679810000001, 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011937237, 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243565+0000", 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0500000000000003e-07 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243585+0000", 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0313e-05 2024-09-15T01:32:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243689+0000", 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103977 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243696+0000", 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0489999999999998e-06 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243791+0000", 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4394000000000005e-05 2024-09-15T01:32:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243800+0000", 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2439999999999999e-06 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244285+0000", 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000484802 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244313+0000", 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8515000000000001e-05 2024-09-15T01:32:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244359+0000", 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5389000000000002e-05 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244375+0000", 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6699000000000001e-05 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.255502+0000", 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011126250000000001 2024-09-15T01:32:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s1 44/31 ECSubWrite(tid=30, reqid=osd.5.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243919+0000", 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4939133600000001, 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010421095999999999, 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243919+0000", 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243919+0000", 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243920+0000", 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2399999999999997e-07 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243929+0000", 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8170000000000005e-06 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245823+0000", 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001894792 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245831+0000", 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9160000000000001e-06 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245883+0000", 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789000000000002e-05 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245894+0000", 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0658e-05 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246000+0000", 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105809 2024-09-15T01:32:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246019+0000", 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250999999999999e-05 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246056+0000", 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6636e-05 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249628+0000", 2024-09-15T01:32:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035721339999999998 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.254340+0000", 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047124699999999999 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.9s2 44/31 ECSubWrite(tid=64, reqid=osd.14.0:60, at_version=0'0, trim_to=0'0, roll_forward_to=44'18))", 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.245393+0000", 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4924389749999998, 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087638879999999992, 2024-09-15T01:32:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245393+0000", 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245393+0000", 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245394+0000", 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3900000000000003e-07 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245406+0000", 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2092e-05 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245442+0000", 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5505e-05 2024-09-15T01:32:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245454+0000", 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1549000000000001e-05 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245508+0000", 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4781999999999998e-05 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245519+0000", 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0497e-05 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245624+0000", 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105402 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245646+0000", 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1739e-05 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245687+0000", 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0728e-05 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249592+0000", 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039053099999999999 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.254157+0000", 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045653450000000002 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 44/31 ECSubWrite(tid=36, reqid=osd.7.0:36, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.246398+0000", 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4914339669999999, 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0094625609999999995, 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246398+0000", 2024-09-15T01:32:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246398+0000", 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246399+0000", 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8e-07 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246422+0000", 2024-09-15T01:32:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3703e-05 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247337+0000", 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091461400000000001 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247344+0000", 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1570000000000003e-06 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247394+0000", 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9966000000000002e-05 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247403+0000", 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7260000000000008e-06 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247834+0000", 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043114100000000001 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5984000000000001e-05 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247900+0000", 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9941999999999998e-05 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.250749+0000", 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028491440000000001 2024-09-15T01:32:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.255861+0000", 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051120039999999999 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.4s4 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.251451+0000", 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.486381878, 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085529130000000005, 2024-09-15T01:32:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251451+0000", 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251451+0000", 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251451+0000", 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0599999999999999e-07 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251474+0000", 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2266999999999999e-05 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251497+0000", 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.336e-05 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0169999999999999e-06 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251554+0000", 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0251999999999999e-05 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251563+0000", 2024-09-15T01:32:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8440000000000004e-06 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251810+0000", 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024663300000000001 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251841+0000", 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0907000000000001e-05 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251894+0000", 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3436999999999997e-05 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251908+0000", 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4132e-05 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.260003+0000", 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080952579999999993 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 44/31 ECSubWrite(tid=39, reqid=client.4477.0:283, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4695459909999999, 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012672954, 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268287+0000", 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.85e-07 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268806+0000", 2024-09-15T01:32:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00051859699999999998 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272018+0000", 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032124620000000001 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272032+0000", 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4008e-05 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272076+0000", 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3807999999999998e-05 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272085+0000", 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9749999999999996e-06 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272331+0000", 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024564600000000003 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272349+0000", 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7988000000000001e-05 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272412+0000", 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2718000000000005e-05 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272426+0000", 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.280959+0000", 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085329199999999994 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:293 3.6s0 3:65d6c4b7:::smithi17529310-30:head [create,setxattr dgeSiblb`YZWoafOlamThlmYtW[NsgYatRkY]Rkn (17332) in=40b,setxattr c`hS`asrnTY^]RZfhhpk^WaRiegkSORj]tZRqnbU (945) in=40b,setxattr biPi^V_QgOq`^qVaOnjr`odUZpilfdNcYZaf]N]q (594) in=40b,setxattr ilZ^tXpioaqcTlZtZR[RiVbNk[cWS^Q`oYjaj_j[ (17824) in=40b,setxattr _p_ks]sbj_gRlVjZr]OjWXZ]gf (1718) in=26b,setxattr eQ\\\\cjZhQZanRhQZcRjiTbSiomgf]n`[kNbokP^` (4560) in=40b,setxattr gqXgeRc[[^pjV]NraefeRN]mbW\\eSlcSOXdPrmSb (1823) in=40b,setxattr Y]hO^[akiVr`]YhQ[oTnksOSSdslk[h\\idalij_d (701) in=40b,setxattr VqPasiXWWe^eT\\Q`e`__hRmg]O_kYeg_mOrmbaaN (1051) in=40b,setxattr ]]i\\^cfhf_mbRNmfWcpO^Tf[Sa`abbsR_el\\OTgR (1366) in=40b,setxattr e]a_`g\\Of\\ipZOgP]aadXoRp\\q^kh_cpolQ[RjXf (1402) in=40b,setxattr VTaQUZ_lrfT`VmUqPmbhXZciq[kkhXT`nkTblbh` (1298) in=40b,setxattr aUjlO\\kVcZYT^ZU\\jbkfd_VRRa]nqn_[T\\ZhlZp[ (14664) in=40b,setxattr sZScfemtX`pkqf^`dZ]ap (1718) in=21b,setxattr agej`oOViShjhSZSRaYRa_k`gk^ecQqfqgOjpa[q (33) in=40b,setxattr fsad_S`\\kPa[[i[VVsaQOZXpPa]pp\\fOQ`tbY`Qg (13472) in=40b,setxattr c\\g]ORfZrTgatTVmX^dSZtTYaXN^YVoZX[Sajonf (1928) in=40b,setxattr T`^]hVNaYYjRdaNne[XsYdaqR]SSTVTsmUnWXqnp (911) in=40b,setxattr ]rWN_b^TgarYNoqUP]qjm_kNbNiPbpth^PrblPjb (315) in=40b,setxattr XtYoarpQjhPgloXSNSob_agQ\\WY\\\\nThcs[XqPZP (1857) in=40b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.269581+0000", 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4682517649999998, 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088342349999999993, 2024-09-15T01:32:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 293 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269581+0000", 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269581+0000", 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269583+0000", 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269783+0000", 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020008499999999999 2024-09-15T01:32:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269799+0000", 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.611e-05 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269805+0000", 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1739999999999997e-06 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269863+0000", 2024-09-15T01:32:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8013000000000002e-05 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269986+0000", 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012216500000000001 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271038+0000", 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001052313 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271289+0000", 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025120200000000001 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271335+0000", 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5512000000000001e-05 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271424+0000", 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9284999999999996e-05 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271758+0000", 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033430299999999999 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.276945+0000", 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051865039999999998 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.278415+0000", 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001470138 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:49.855 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:49.856 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:32:49.857 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:32:49.857 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:32:49.857 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:32:49.858 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:32:49.858 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:32:49.858 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:32:49.858 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:32:49.859 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:32:49.859 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:32:49.859 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:32:49.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:32:49.845+0000 7faeb1aef640 0 -- 172.21.15.42:0/3677355563 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7faeac194950 msgr2=0x7faeac196d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:32:49.860 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:32:49.860 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:49.855+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:49.872 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:32:49.872 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:32:49.897 INFO:tasks.rados.rados.0.smithi175.stdout:133: finishing rollback tid 2 to smithi17529310-17 2024-09-15T01:32:49.897 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 17 v 51 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-15T01:32:49.957 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:49.955+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:49.973 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:49.973 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:49.973 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:32:49.973 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:49.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:32:49.983 INFO:tasks.rados.rados.0.smithi175.stdout:143: finishing write tid 4 to smithi17529310-19 2024-09-15T01:32:49.984 INFO:tasks.rados.rados.0.smithi175.stdout:143: finishing write tid 5 to smithi17529310-19 2024-09-15T01:32:49.984 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 19 v 25 (ObjNum 87 snap 7 seq_num 87) dirty exists 2024-09-15T01:32:49.985 INFO:tasks.rados.rados.0.smithi175.stdout:143: left oid 19 (ObjNum 87 snap 7 seq_num 87) 2024-09-15T01:32:50.065 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:50.064+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:50.094 INFO:tasks.rados.rados.0.smithi175.stdout:144: finishing write tid 4 to smithi17529310-45 2024-09-15T01:32:50.095 INFO:tasks.rados.rados.0.smithi175.stdout:144: finishing write tid 5 to smithi17529310-45 2024-09-15T01:32:50.095 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 45 v 9 (ObjNum 88 snap 7 seq_num 88) dirty exists 2024-09-15T01:32:50.095 INFO:tasks.rados.rados.0.smithi175.stdout:144: left oid 45 (ObjNum 88 snap 7 seq_num 88) 2024-09-15T01:32:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:32:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:32:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:50.131 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:32:50.172 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:50.171+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:50.191 INFO:tasks.rados.rados.0.smithi175.stdout:142: finishing write tid 4 to smithi17529310-32 2024-09-15T01:32:50.193 INFO:tasks.rados.rados.0.smithi175.stdout:142: finishing write tid 5 to smithi17529310-32 2024-09-15T01:32:50.193 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 32 v 39 (ObjNum 86 snap 7 seq_num 86) dirty exists 2024-09-15T01:32:50.193 INFO:tasks.rados.rados.0.smithi175.stdout:142: left oid 32 (ObjNum 86 snap 7 seq_num 86) 2024-09-15T01:32:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-15T01:32:50.259 INFO:tasks.rados.rados.0.smithi175.stdout:133: done (8 left) 2024-09-15T01:32:50.259 INFO:tasks.rados.rados.0.smithi175.stdout:134: done (7 left) 2024-09-15T01:32:50.260 INFO:tasks.rados.rados.0.smithi175.stdout:141: done (6 left) 2024-09-15T01:32:50.260 INFO:tasks.rados.rados.0.smithi175.stdout:142: done (5 left) 2024-09-15T01:32:50.260 INFO:tasks.rados.rados.0.smithi175.stdout:143: done (4 left) 2024-09-15T01:32:50.260 INFO:tasks.rados.rados.0.smithi175.stdout:144: done (3 left) 2024-09-15T01:32:50.260 INFO:tasks.rados.rados.0.smithi175.stdout:146: done (2 left) 2024-09-15T01:32:50.260 INFO:tasks.rados.rados.0.smithi175.stdout:147: done (1 left) 2024-09-15T01:32:50.260 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 2 current snap is 8 2024-09-15T01:32:50.260 INFO:tasks.rados.rados.0.smithi175.stdout:148: seq_num 90 ranges {32768=753664,786432=720896,1507328=49152} 2024-09-15T01:32:50.260 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:50.258+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:50.264 INFO:tasks.rados.rados.0.smithi175.stdout:148: writing smithi17529310-2 from 32768 to 786432 tid 1 2024-09-15T01:32:50.269 INFO:tasks.rados.rados.0.smithi175.stdout:148: writing smithi17529310-2 from 786432 to 1507328 tid 2 2024-09-15T01:32:50.269 INFO:tasks.rados.rados.0.smithi175.stdout:148: writing smithi17529310-2 from 1507328 to 1556480 tid 3 2024-09-15T01:32:50.269 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 34 current snap is 8 2024-09-15T01:32:50.269 INFO:tasks.rados.rados.0.smithi175.stdout:149: seq_num 91 ranges {557056=770048,1327104=475136,1802240=294912} 2024-09-15T01:32:50.271 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-15T01:32:50.276 INFO:tasks.rados.rados.0.smithi175.stdout:149: writing smithi17529310-34 from 557056 to 1327104 tid 1 2024-09-15T01:32:50.278 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:50.278 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:32:50.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:32:50.278 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:32:50.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.278 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:14 3.1s0 3:8930608f:::smithi17529310-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:50.278 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.176805+0000", 2024-09-15T01:32:50.278 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.098489775999999, 2024-09-15T01:32:50.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020710388, 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 14 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176805+0000", 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176805+0000", 2024-09-15T01:32:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176806+0000", 2024-09-15T01:32:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-15T01:32:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176810+0000", 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8399999999999997e-06 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176824+0000", 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4051e-05 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176827+0000", 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5159999999999999e-06 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178153+0000", 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013253519999999999 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178241+0000", 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8670000000000003e-05 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178708+0000", 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00046673299999999999 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178860+0000", 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015201099999999999 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179098+0000", 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238123 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179138+0000", 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0184e-05 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180215+0000", 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001076544 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197335+0000", 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017120209000000001 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197516+0000", 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000180657 2024-09-15T01:32:50.284 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:15 3.1s0 3:8930608f:::smithi17529310-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.176858+0000", 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.098436689, 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023897231000000001, 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.285 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176858+0000", 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176858+0000", 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176859+0000", 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0999999999999999e-07 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176861+0000", 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4099999999999998e-06 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176875+0000", 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3366e-05 2024-09-15T01:32:50.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176882+0000", 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5560000000000002e-06 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178931+0000", 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002048299 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179014+0000", 2024-09-15T01:32:50.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3207000000000005e-05 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197559+0000", 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018545582000000001 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197656+0000", 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7085999999999998e-05 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.200756+0000", 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030992150000000002 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:29 3.7s0 3:e2b4e7d9:::smithi17529310-10:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.177051+0000", 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.098243742999999, 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020495925000000002, 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 29 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177051+0000", 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177051+0000", 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177052+0000", 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177064+0000", 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1477999999999999e-05 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177072+0000", 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5629999999999992e-06 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177084+0000", 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1557e-05 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178451+0000", 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013673019999999999 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178548+0000", 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6639999999999996e-05 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179029+0000", 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048111300000000003 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179171+0000", 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014193800000000001 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2196e-05 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179212+0000", 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9462999999999999e-05 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179251+0000", 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9155e-05 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197330+0000", 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018078409 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197547+0000", 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021747699999999999 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:30 3.7s0 3:e2b4e7d9:::smithi17529310-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.177099+0000", 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.098196005, 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024179603000000001, 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 30 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177099+0000", 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177099+0000", 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177100+0000", 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-07 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177103+0000", 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.847e-06 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177111+0000", 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4780000000000008e-06 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177116+0000", 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999998e-06 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179311+0000", 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002195331 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179405+0000", 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4611999999999995e-05 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197589+0000", 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018183300999999999 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197665+0000", 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5878999999999996e-05 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.201279+0000", 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003614089 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:31 3.1s0 3:8776790e:::smithi17529310-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.177756+0000", 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.09753937, 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020342740000000002, 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 31 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177756+0000", 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177756+0000", 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177757+0000", 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178059+0000", 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030214800000000002 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178069+0000", 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0324e-05 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178081+0000", 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1491999999999999e-05 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179041+0000", 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096040399999999999 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179333+0000", 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029163799999999999 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180034+0000", 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070128900000000001 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180155+0000", 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120844 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180166+0000", 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0967e-05 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180220+0000", 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4048999999999997e-05 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182483+0000", 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022628930000000002 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.198037+0000", 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015553726 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.198099+0000", 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1824999999999998e-05 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:32 3.1s0 3:8776790e:::smithi17529310-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.178085+0000", 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.097209775, 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021154846000000001, 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178085+0000", 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178085+0000", 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178086+0000", 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0699999999999997e-07 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178088+0000", 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1100000000000001e-06 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178096+0000", 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4419999999999998e-06 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178101+0000", 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1810000000000003e-06 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180259+0000", 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021580839999999998 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180359+0000", 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100028 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181451+0000", 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001092569 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181638+0000", 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018707899999999999 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181720+0000", 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1682999999999997e-05 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182311+0000", 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059133299999999998 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182487+0000", 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017578299999999999 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199166+0000", 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016678622000000001 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199240+0000", 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4425000000000007e-05 2024-09-15T01:32:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:33 3.1s0 3:8776790e:::smithi17529310-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.097190351, 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024100798, 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 33 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:32:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178108+0000", 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3259999999999998e-06 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178114+0000", 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5209999999999999e-06 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178118+0000", 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7139999999999999e-06 2024-09-15T01:32:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182355+0000", 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042371070000000004 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182467+0000", 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011186799999999999 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199900+0000", 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017433015999999999 2024-09-15T01:32:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199973+0000", 2024-09-15T01:32:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2897000000000004e-05 2024-09-15T01:32:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.202206+0000", 2024-09-15T01:32:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022329049999999999 2024-09-15T01:32:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:44 3.bs0 3:d6f740a8:::smithi17529310-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.185918+0000", 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.089377304999999, 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.123698503, 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185918+0000", 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185918+0000", 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185919+0000", 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7400000000000003e-07 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185927+0000", 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7860000000000002e-06 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185936+0000", 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7569999999999993e-06 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185941+0000", 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1889999999999998e-06 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187237+0000", 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001295806 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187337+0000", 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9630000000000007e-05 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187837+0000", 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000500757 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187919+0000", 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1482000000000004e-05 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187962+0000", 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2647e-05 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188001+0000", 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9070999999999999e-05 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188078+0000", 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7713999999999995e-05 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309538+0000", 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.121459186 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309616+0000", 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8804e-05 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:45 3.bs0 3:d6f740a8:::smithi17529310-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.089349531, 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12623172299999999, 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 45 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185948+0000", 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185954+0000", 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1090000000000003e-06 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185958+0000", 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5159999999999999e-06 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187975+0000", 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002017343 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188054+0000", 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8734999999999997e-05 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309670+0000", 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.121615926 2024-09-15T01:32:50.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309756+0000", 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6316999999999994e-05 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.312177+0000", 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002421126 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.328 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:77 3.7s0 3:eb933b0c:::smithi17529310-26:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.213552+0000", 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.061742805, 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017819044999999999, 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 77 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213552+0000", 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213552+0000", 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213553+0000", 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7899999999999997e-07 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213558+0000", 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6489999999999999e-06 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213567+0000", 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6130000000000004e-06 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213571+0000", 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1740000000000002e-06 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.214963+0000", 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013925350000000001 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215037+0000", 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4158000000000004e-05 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215424+0000", 2024-09-15T01:32:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038638099999999999 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215508+0000", 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3863999999999995e-05 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215521+0000", 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3089e-05 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215553+0000", 2024-09-15T01:32:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2001000000000001e-05 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215598+0000", 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5510999999999998e-05 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231298+0000", 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015699856000000002 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231371+0000", 2024-09-15T01:32:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3335000000000002e-05 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:78 3.7s0 3:eb933b0c:::smithi17529310-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.213590+0000", 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.061704957, 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020370672999999999, 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 78 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213590+0000", 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213590+0000", 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213591+0000", 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213593+0000", 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3199999999999998e-06 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213600+0000", 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4580000000000004e-06 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213607+0000", 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-06 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215631+0000", 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020248890000000002 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215718+0000", 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6127999999999999e-05 2024-09-15T01:32:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231591+0000", 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015873015000000001 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231666+0000", 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5525999999999995e-05 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233961+0000", 2024-09-15T01:32:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022948069999999998 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:93 3.7s0 3:e6e8875f:::smithi17529310-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.045909623, 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079343245000000007, 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 93 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1299999999999999e-07 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229390+0000", 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.703e-06 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229403+0000", 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3056e-05 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229410+0000", 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231135+0000", 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001725737 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231216+0000", 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0586000000000004e-05 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233251+0000", 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020346819999999999 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233339+0000", 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7907999999999999e-05 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308729+0000", 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.075390190999999995 2024-09-15T01:32:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.0s6 44/31 ECSubWrite(tid=62, reqid=osd.14.0:62, at_version=44'24, trim_to=0'0, roll_forward_to=43'20))", 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.240097+0000", 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0351986469999996, 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011291046000000001, 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240097+0000", 2024-09-15T01:32:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240097+0000", 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240098+0000", 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8610000000000001e-06 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240131+0000", 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.218e-05 2024-09-15T01:32:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240175+0000", 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3992999999999997e-05 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240182+0000", 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5059999999999998e-06 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240220+0000", 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7718e-05 2024-09-15T01:32:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240230+0000", 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0589e-05 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240463+0000", 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023241099999999999 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240490+0000", 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6945e-05 2024-09-15T01:32:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240525+0000", 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5388000000000001e-05 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240541+0000", 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6121e-05 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251388+0000", 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010846333999999999 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.3s2 44/31 ECSubWrite(tid=152, reqid=osd.13.0:152, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243723+0000", 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0315719789999998, 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0093291339999999993, 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243723+0000", 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243723+0000", 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243724+0000", 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4600000000000004e-07 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243756+0000", 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1912999999999998e-05 2024-09-15T01:32:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243797+0000", 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1510000000000001e-05 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243804+0000", 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7499999999999997e-06 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244011+0000", 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020705100000000001 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244020+0000", 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3049999999999998e-06 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244238+0000", 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021728000000000001 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244280+0000", 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.197e-05 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244333+0000", 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3502999999999999e-05 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244722+0000", 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038922300000000001 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253052+0000", 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083298829999999997 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds6 44/31 ECSubWrite(tid=36, reqid=osd.7.0:36, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.246189+0000", 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0291060359999999, 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080732459999999992, 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246189+0000", 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246189+0000", 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246189+0000", 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9600000000000001e-07 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246209+0000", 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0166e-05 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246281+0000", 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1353999999999995e-05 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246285+0000", 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6240000000000001e-06 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246539+0000", 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025348800000000001 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246549+0000", 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5519999999999993e-06 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246751+0000", 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020273399999999999 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246766+0000", 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4442e-05 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246814+0000", 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7936999999999999e-05 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246926+0000", 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112698 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.254262+0000", 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0073360550000000002 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds6 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.246995+0000", 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0283002889999997, 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087697509999999992, 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.360 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246995+0000", 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246995+0000", 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246995+0000", 2024-09-15T01:32:50.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000003e-07 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247025+0000", 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9292e-05 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247061+0000", 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247073+0000", 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1420000000000001e-05 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247128+0000", 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5083000000000001e-05 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247136+0000", 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7870000000000002e-06 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247404+0000", 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026824999999999999 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247432+0000", 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7929999999999999e-05 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247463+0000", 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1013999999999998e-05 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247473+0000", 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0715e-05 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.255765+0000", 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082911459999999992 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:306 3.7s0 3:e6e8875f:::smithi17529310-31:head [append 0~0] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:32:50.365 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.269780+0000", 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0055155539999996, 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013438767000000001, 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 306 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269780+0000", 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269780+0000", 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269782+0000", 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3489999999999999e-06 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269788+0000", 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1589999999999998e-06 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269806+0000", 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7642000000000002e-05 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269813+0000", 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6159999999999996e-06 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269878+0000", 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4121999999999994e-05 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270069+0000", 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000191769 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270780+0000", 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071044299999999995 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271660+0000", 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00088018600000000003 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271671+0000", 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1258e-05 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271715+0000", 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4091000000000002e-05 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272505+0000", 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00078956700000000002 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.283171+0000", 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010666261 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.283218+0000", 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7304e-05 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:307 3.7s0 3:e6e8875f:::smithi17529310-31:head [rollback 8] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.270030+0000", 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.005265337, 2024-09-15T01:32:50.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013327521, 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 307 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270030+0000", 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270030+0000", 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270031+0000", 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9000000000000005e-07 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270035+0000", 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1300000000000003e-06 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270046+0000", 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0661e-05 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270054+0000", 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0039999999999999e-06 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271724+0000", 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001670232 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271818+0000", 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4290999999999996e-05 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272296+0000", 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047796699999999999 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272405+0000", 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109005 2024-09-15T01:32:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272420+0000", 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5184000000000001e-05 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272460+0000", 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9787999999999999e-05 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272511+0000", 2024-09-15T01:32:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0872999999999999e-05 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.283309+0000", 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010797676000000001 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.283357+0000", 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8720000000000001e-05 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:308 3.7s0 3:e6e8875f:::smithi17529310-31:head [append 0~0] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.270062+0000", 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0052327310000004, 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013400334, 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 308 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270062+0000", 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270062+0000", 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270063+0000", 2024-09-15T01:32:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6100000000000001e-07 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270065+0000", 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5909999999999998e-06 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270073+0000", 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9680000000000006e-06 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270096+0000", 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2739e-05 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272644+0000", 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025474 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272741+0000", 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7428000000000003e-05 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.273173+0000", 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043247400000000002 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.273245+0000", 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1952000000000005e-05 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.274248+0000", 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010024890000000001 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.274289+0000", 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0633000000000002e-05 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.274306+0000", 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7827000000000001e-05 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.283424+0000", 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091179590000000001 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.283463+0000", 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8413e-05 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.7s0 47/38 ECSubWriteReply(tid=76, last_complete=47'25, committed=1, applied=1))", 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.91107261399999995, 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.42564722900000002, 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:32:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8599999999999999e-07 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364224+0000", 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5099999999999999e-06 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789796+0000", 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.425571957 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789805+0000", 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789857+0000", 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1598999999999998e-05 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789870+0000", 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2850000000000001e-05 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.388 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:50.390 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:32:50.392 INFO:tasks.rados.rados.0.smithi175.stdout:149: writing smithi17529310-34 from 1327104 to 1802240 tid 2 2024-09-15T01:32:50.392 INFO:tasks.rados.rados.0.smithi175.stdout:149: writing smithi17529310-34 from 1802240 to 2097152 tid 3 2024-09-15T01:32:50.392 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 3 current snap is 8 2024-09-15T01:32:50.392 INFO:tasks.rados.rados.0.smithi175.stdout:148: finishing write tid 1 to smithi17529310-2 2024-09-15T01:32:50.392 INFO:tasks.rados.rados.0.smithi175.stdout:148: finishing write tid 2 to smithi17529310-2 2024-09-15T01:32:50.392 INFO:tasks.rados.rados.0.smithi175.stdout:148: finishing write tid 3 to smithi17529310-2 2024-09-15T01:32:50.392 INFO:tasks.rados.rados.0.smithi175.stdout:150: seq_num 92 ranges {1048576=786432,1835008=491520,2326528=278528} 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:150: writing smithi17529310-3 from 1048576 to 1835008 tid 1 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:150: writing smithi17529310-3 from 1835008 to 2326528 tid 2 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:150: writing smithi17529310-3 from 2326528 to 2605056 tid 3 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:151: copy_from oid 38 from oid 50 current snap is 8 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:148: finishing write tid 4 to smithi17529310-2 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:148: finishing write tid 5 to smithi17529310-2 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 2 v 56 (ObjNum 90 snap 8 seq_num 90) dirty exists 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:148: left oid 2 (ObjNum 90 snap 8 seq_num 90) 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:145: finishing write tid 4 to smithi17529310-30 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:145: finishing write tid 5 to smithi17529310-30 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 30 v 21 (ObjNum 89 snap 7 seq_num 89) dirty exists 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:145: left oid 30 (ObjNum 89 snap 7 seq_num 89) 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:149: finishing write tid 1 to smithi17529310-34 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:149: finishing write tid 2 to smithi17529310-34 2024-09-15T01:32:50.393 INFO:tasks.rados.rados.0.smithi175.stdout:149: finishing write tid 3 to smithi17529310-34 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:149: finishing write tid 4 to smithi17529310-34 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:149: finishing write tid 5 to smithi17529310-34 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 34 v 14 (ObjNum 91 snap 8 seq_num 91) dirty exists 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:149: left oid 34 (ObjNum 91 snap 8 seq_num 91) 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:145: done (4 left) 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:148: done (3 left) 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:149: done (2 left) 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:152: snap_create 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:150: finishing write tid 1 to smithi17529310-3 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:150: finishing write tid 2 to smithi17529310-3 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:150: finishing write tid 3 to smithi17529310-3 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:150: finishing write tid 4 to smithi17529310-3 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:150: finishing write tid 5 to smithi17529310-3 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 3 v 27 (ObjNum 92 snap 8 seq_num 92) dirty exists 2024-09-15T01:32:50.394 INFO:tasks.rados.rados.0.smithi175.stdout:150: left oid 3 (ObjNum 92 snap 8 seq_num 92) 2024-09-15T01:32:50.395 INFO:tasks.rados.rados.0.smithi175.stdout:151: finishing copy_from racing read to smithi17529310-38 2024-09-15T01:32:50.395 INFO:tasks.rados.rados.0.smithi175.stdout:151: finishing copy_from to smithi17529310-38 2024-09-15T01:32:50.395 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 38 v 11 (ObjNum 66 snap 3 seq_num 66) dirty exists 2024-09-15T01:32:50.395 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:50.387+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:50.489 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:50.488+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:50.536 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:32:50.536 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:50.545 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:32:50.590 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:50.588+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:50.646 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:32:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:32:50.646 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:50.654 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:32:50.695 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:50.694+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:50.792 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:50.790+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:50.797 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:32:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:32:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:32:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.5s1 33/31 ECSubWrite(tid=1, reqid=client.4477.0:11, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:32:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.616163024, 2024-09-15T01:32:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030602694, 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8900000000000001e-07 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178311+0000", 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5959999999999993e-06 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208281+0000", 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029970173999999999 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208291+0000", 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3850000000000008e-06 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208407+0000", 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116168 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208422+0000", 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4931e-05 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208519+0000", 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7213999999999995e-05 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208541+0000", 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2249999999999999e-05 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208644+0000", 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102537 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208664+0000", 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.071e-05 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208904+0000", 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023944 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.5s1 34/31 ECSubRead(tid=6, to_read={3:af8bec4d:::smithi17529310-24:head=0,4096,0}, subchunks={3:af8bec4d:::smithi17529310-24:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.210749+0000", 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.583715148, 2024-09-15T01:32:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033380170000000001, 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210749+0000", 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210749+0000", 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210750+0000", 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6000000000000004e-07 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210754+0000", 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7049999999999998e-06 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.243939+0000", 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033185092999999999 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.243951+0000", 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1864e-05 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.244010+0000", 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8536000000000003e-05 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.244129+0000", 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119412 2024-09-15T01:32:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/31 ECSubWrite(tid=22, reqid=client.4477.0:113, at_version=34'10, trim_to=0'0, roll_forward_to=34'7))", 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.539245275000001, 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035418835000000003, 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255233+0000", 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3815000000000001e-05 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.289999+0000", 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.034765375000000001 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290004+0000", 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290062+0000", 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7763999999999998e-05 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290072+0000", 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7829999999999993e-06 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290165+0000", 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3039000000000001e-05 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290192+0000", 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7449000000000001e-05 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290303+0000", 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110253 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290375+0000", 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2855999999999995e-05 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290638+0000", 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026231300000000002 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:128 3.4s0 3:299c4018:::smithi17529310-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.275429+0000", 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.519035173000001, 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.10370515499999999, 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 128 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275429+0000", 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275429+0000", 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275430+0000", 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1099999999999995e-07 2024-09-15T01:32:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275434+0000", 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.42e-06 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275443+0000", 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1330000000000005e-06 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275448+0000", 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3769999999999998e-06 2024-09-15T01:32:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.276441+0000", 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099363500000000009 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.276580+0000", 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013892099999999999 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277288+0000", 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070808500000000001 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277385+0000", 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6824999999999995e-05 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277394+0000", 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0839999999999998e-06 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277434+0000", 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9465999999999998e-05 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277551+0000", 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117162 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379048+0000", 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101497254 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379134+0000", 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6081999999999993e-05 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:129 3.4s0 3:299c4018:::smithi17529310-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.518995331999999, 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.105118901, 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 129 2024-09-15T01:32:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:32:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8999999999999996e-07 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275472+0000", 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6419999999999999e-06 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275480+0000", 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0790000000000001e-06 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275485+0000", 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4689999999999999e-06 2024-09-15T01:32:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277457+0000", 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019728160000000001 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277534+0000", 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6073000000000002e-05 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379464+0000", 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.10193010499999999 2024-09-15T01:32:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379556+0000", 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2084999999999997e-05 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.380588+0000", 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010320419999999999 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.cs1 34/31 ECSubRead(tid=37, to_read={3:34a45b5f:::smithi17529310-48:head=0,4096,0}, subchunks={3:34a45b5f:::smithi17529310-48:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:32:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.486629518000001, 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14311833299999999, 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6499999999999999e-07 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307844+0000", 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4810000000000006e-06 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450796+0000", 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14295194899999999 2024-09-15T01:32:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450806+0000", 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7890000000000006e-06 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450858+0000", 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2577999999999997e-05 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450953+0000", 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4870999999999993e-05 2024-09-15T01:32:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:217 3.4s0 3:299c4018:::smithi17529310-43:head [copy-from ver 4] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.208686+0000", 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.5857780990000006, 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013624864, 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 217 2024-09-15T01:32:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208686+0000", 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208686+0000", 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208688+0000", 2024-09-15T01:32:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3989999999999998e-06 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208709+0000", 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0849999999999999e-05 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208729+0000", 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9681e-05 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208738+0000", 2024-09-15T01:32:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5350000000000004e-06 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208841+0000", 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103353 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208941+0000", 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9986000000000001e-05 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.219554+0000", 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01061337 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.219922+0000", 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036728800000000001 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.219935+0000", 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3689000000000001e-05 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220006+0000", 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1073000000000002e-05 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220034+0000", 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7736e-05 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.221399+0000", 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013649000000000001 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.222311+0000", 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000912004 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:218 3.4s0 3:299c4018:::smithi17529310-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.5857115630000003, 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011546681, 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 218 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:32:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6799999999999999e-07 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208757+0000", 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.941e-06 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208767+0000", 2024-09-15T01:32:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8509999999999994e-06 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208786+0000", 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8320000000000001e-05 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209130+0000", 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034496100000000002 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209184+0000", 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3703999999999999e-05 2024-09-15T01:32:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220077+0000", 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010893277999999999 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220195+0000", 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117342 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220299+0000", 2024-09-15T01:32:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104516 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.9s5 44/31 ECSubWrite(tid=60, reqid=osd.14.0:60, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.238474+0000", 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5559902599999997, 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010306067, 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.238474+0000", 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.238474+0000", 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.238474+0000", 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.238499+0000", 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4298999999999999e-05 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.238558+0000", 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9790999999999997e-05 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.238570+0000", 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2001000000000001e-05 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.239089+0000", 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00051868599999999997 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.239101+0000", 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1545999999999999e-05 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240653+0000", 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015523399999999999 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240922+0000", 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026856699999999999 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241196+0000", 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027480400000000002 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241239+0000", 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2919e-05 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248780+0000", 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075407540000000002 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs1 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.241172+0000", 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5532918469999997, 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018176115, 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241172+0000", 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241172+0000", 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241173+0000", 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.05e-07 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241195+0000", 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2033000000000001e-05 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241238+0000", 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3507999999999997e-05 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241245+0000", 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2509999999999997e-06 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245590+0000", 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043445309999999996 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245599+0000", 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8519999999999993e-06 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246349+0000", 2024-09-15T01:32:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00075022999999999995 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246362+0000", 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.273e-05 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246399+0000", 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7299000000000003e-05 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246806+0000", 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040714800000000003 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.259348+0000", 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012542328 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs1 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:32:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5511986049999997, 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017149162999999999, 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243294+0000", 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7868000000000002e-05 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243334+0000", 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9369000000000002e-05 2024-09-15T01:32:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243341+0000", 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7009999999999996e-06 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246522+0000", 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031802359999999999 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246530+0000", 2024-09-15T01:32:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5019999999999992e-06 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246730+0000", 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019988499999999999 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246745+0000", 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4632e-05 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246784+0000", 2024-09-15T01:32:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9072000000000001e-05 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246850+0000", 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699000000000006e-05 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.260415+0000", 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013564296 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 44/31 ECSubWrite(tid=82, reqid=client.4477.0:281, at_version=44'36, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.244090+0000", 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5503739100000002, 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0099664520000000006, 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244090+0000", 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244090+0000", 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244091+0000", 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244125+0000", 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3414000000000002e-05 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244173+0000", 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8111000000000001e-05 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244180+0000", 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2010000000000002e-06 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244245+0000", 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4628999999999994e-05 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244254+0000", 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3340000000000008e-06 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245520+0000", 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012662210000000001 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245531+0000", 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.03e-05 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245586+0000", 2024-09-15T01:32:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5575999999999998e-05 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245878+0000", 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029204099999999999 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.254057+0000", 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081785069999999998 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as3 44/38 ECSubWrite(tid=25, reqid=osd.3.0:25, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.244320+0000", 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5501445199999999, 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010577385999999999, 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244320+0000", 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244320+0000", 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244320+0000", 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244344+0000", 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4134e-05 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244427+0000", 2024-09-15T01:32:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2767000000000002e-05 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244434+0000", 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6610000000000003e-06 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244924+0000", 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049050600000000004 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244934+0000", 2024-09-15T01:32:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1889999999999996e-06 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246070+0000", 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001135973 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246098+0000", 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.809e-05 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246133+0000", 2024-09-15T01:32:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5670000000000002e-05 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246146+0000", 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3141e-05 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.254897+0000", 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087505680000000002 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 44/31 ECSubWrite(tid=83, reqid=osd.11.0:83, at_version=44'38, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.245401+0000", 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5490627049999999, 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0098909730000000008, 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245401+0000", 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245401+0000", 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245402+0000", 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2399999999999997e-07 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245424+0000", 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1650000000000001e-05 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245456+0000", 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2016e-05 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245462+0000", 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.964e-06 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245579+0000", 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117553 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245589+0000", 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1370000000000008e-06 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245842+0000", 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025340999999999997 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245853+0000", 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0712e-05 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245924+0000", 2024-09-15T01:32:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1120999999999999e-05 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245935+0000", 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1208e-05 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.255292+0000", 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0093574780000000007 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds5 44/31 ECSubWrite(tid=36, reqid=osd.7.0:36, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5484736039999998, 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013253133, 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6199999999999999e-07 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246013+0000", 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1801e-05 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246123+0000", 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109879 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246132+0000", 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8670000000000006e-06 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246240+0000", 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107211 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246248+0000", 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8480000000000007e-06 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246452+0000", 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020319399999999999 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246461+0000", 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246492+0000", 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1124999999999998e-05 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246831+0000", 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033878400000000001 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.259244+0000", 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012412444 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.246909+0000", 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5475555759999997, 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013946906, 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246909+0000", 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246909+0000", 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246910+0000", 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.16e-07 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246933+0000", 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3884e-05 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246981+0000", 2024-09-15T01:32:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7194000000000002e-05 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246988+0000", 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0260000000000002e-06 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247055+0000", 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7216000000000002e-05 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247066+0000", 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1562999999999999e-05 2024-09-15T01:32:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248672+0000", 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001605455 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248697+0000", 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5588e-05 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248730+0000", 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2351000000000002e-05 2024-09-15T01:32:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248812+0000", 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2254000000000002e-05 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.260856+0000", 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012043458999999999 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds5 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5473376889999999, 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014066664, 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4000000000000001e-07 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247148+0000", 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1339e-05 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247188+0000", 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9301000000000002e-05 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247196+0000", 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2789999999999998e-06 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247453+0000", 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025692899999999998 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247463+0000", 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0317e-05 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248725+0000", 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012613730000000001 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248732+0000", 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9129999999999997e-06 2024-09-15T01:32:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248776+0000", 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4280000000000003e-05 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248811+0000", 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5553000000000003e-05 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.261193+0000", 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01238174 2024-09-15T01:32:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as3 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5466044380000001, 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012019658000000001, 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247861+0000", 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6700000000000002e-07 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247872+0000", 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1307e-05 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247909+0000", 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7290999999999999e-05 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247917+0000", 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3590000000000002e-06 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248032+0000", 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115566 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248048+0000", 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6069000000000001e-05 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248184+0000", 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013525399999999999 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248205+0000", 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1021000000000001e-05 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248244+0000", 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9847e-05 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248260+0000", 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5639000000000001e-05 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.259879+0000", 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011619338 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds5 44/31 ECSubWrite(tid=38, reqid=osd.7.0:37, at_version=0'0, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.251858+0000", 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5426061319999995, 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010171173, 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251858+0000", 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251858+0000", 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251943+0000", 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4887000000000002e-05 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252362+0000", 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0004193 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252452+0000", 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0026999999999997e-05 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252460+0000", 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0080000000000002e-06 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252528+0000", 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7414000000000002e-05 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252538+0000", 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.023e-05 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252674+0000", 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000136017 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252704+0000", 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0368000000000001e-05 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253070+0000", 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000365204 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253092+0000", 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2693000000000001e-05 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.262029+0000", 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089370249999999995 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:292 3.4s0 3:299c4018:::smithi17529310-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.267708+0000", 2024-09-15T01:32:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5267562980000005, 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020077541000000001, 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 292 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267708+0000", 2024-09-15T01:32:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267708+0000", 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267709+0000", 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7800000000000001e-07 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267712+0000", 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267723+0000", 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1388e-05 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267729+0000", 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5890000000000002e-06 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269684+0000", 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019553389999999999 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269888+0000", 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020362599999999999 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.273250+0000", 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033616280000000002 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.273338+0000", 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7990000000000003e-05 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.287785+0000", 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01444779 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:50.897 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:50.899 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-15T01:32:50.899 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:50.897+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:50.999 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:50.998+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:51.000 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:51.000 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:51.000 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:32:51.000 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:51.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-15T01:32:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.8 to 1 (10000) 2024-09-15T01:32:51.069 INFO:tasks.rados.rados.0.smithi175.stdout:150: done (2 left) 2024-09-15T01:32:51.069 INFO:tasks.rados.rados.0.smithi175.stdout:151: done (1 left) 2024-09-15T01:32:51.069 INFO:tasks.rados.rados.0.smithi175.stdout:152: done (0 left) 2024-09-15T01:32:51.069 INFO:tasks.rados.rados.0.smithi175.stdout:153: snap_remove snap 5 2024-09-15T01:32:51.080 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-15T01:32:51.111 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:32:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:32:51.111 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:51.111 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:51.109+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:51.120 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-15T01:32:51.201 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:51.200+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:51.269 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:51.269 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:32:51.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:32:51.269 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:32:51.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.269 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 34/31 ECSubWrite(tid=3, reqid=client.4477.0:37, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:51.269 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.183485+0000", 2024-09-15T01:32:51.269 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.082560628, 2024-09-15T01:32:51.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024283960000000002, 2024-09-15T01:32:51.269 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.269 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.183485+0000", 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.183485+0000", 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.183485+0000", 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-07 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.183758+0000", 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027313000000000001 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.183801+0000", 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2811000000000001e-05 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.183809+0000", 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3469999999999999e-06 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185090+0000", 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012812609999999999 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185099+0000", 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6240000000000008e-06 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185237+0000", 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013811699999999999 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185267+0000", 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0332999999999999e-05 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185483+0000", 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021506800000000001 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185530+0000", 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7929999999999997e-05 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185913+0000", 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038255800000000002 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 34/31 ECSubWrite(tid=6, reqid=client.4477.0:43, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:51.274 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.186998+0000", 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.079047021999999, 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021211789999999999, 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.186998+0000", 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.186998+0000", 2024-09-15T01:32:51.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.186999+0000", 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187073+0000", 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4532999999999999e-05 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188459+0000", 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001385568 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188467+0000", 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1130000000000001e-06 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188552+0000", 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5385999999999999e-05 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188565+0000", 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3309000000000001e-05 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188713+0000", 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147362 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188744+0000", 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1628999999999998e-05 2024-09-15T01:32:51.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188808+0000", 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3998999999999995e-05 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188836+0000", 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7392000000000001e-05 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.189119+0000", 2024-09-15T01:32:51.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028352100000000001 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 34/31 ECSubWrite(tid=7, reqid=client.4477.0:44, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.077622605, 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.119995086, 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188439+0000", 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5613000000000001e-05 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307929+0000", 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.119490443 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307936+0000", 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1740000000000003e-06 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308009+0000", 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2714999999999998e-05 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308018+0000", 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8019999999999998e-06 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308149+0000", 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130739 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308161+0000", 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2481e-05 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308199+0000", 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7492e-05 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308216+0000", 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7076999999999999e-05 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308418+0000", 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020204899999999999 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 34/31 ECSubWrite(tid=23, reqid=client.4477.0:150, at_version=34'3, trim_to=0'0, roll_forward_to=34'1))", 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.307819+0000", 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.958225936, 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002233325, 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307819+0000", 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307819+0000", 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307820+0000", 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0699999999999996e-07 2024-09-15T01:32:51.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307903+0000", 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3479000000000005e-05 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309400+0000", 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001496163 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309407+0000", 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0840000000000003e-06 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309454+0000", 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7688000000000001e-05 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309463+0000", 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8140000000000007e-06 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309590+0000", 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012654100000000001 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309616+0000", 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6322999999999999e-05 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309716+0000", 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9449000000000003e-05 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309734+0000", 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8241e-05 2024-09-15T01:32:51.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.310053+0000", 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031883600000000002 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 37/36 ECSubWrite(tid=30, reqid=client.4477.0:185, at_version=37'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:39.189271+0000", 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.076773803, 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002302644, 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.290 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.189271+0000", 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.189271+0000", 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.189272+0000", 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3600000000000003e-07 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.189301+0000", 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9011999999999999e-05 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.189345+0000", 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3761999999999999e-05 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.189353+0000", 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7959999999999996e-06 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.189405+0000", 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1857999999999997e-05 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.189419+0000", 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4812e-05 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.189673+0000", 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025381200000000002 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.189685+0000", 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1695000000000001e-05 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.189725+0000", 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0073000000000001e-05 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.189750+0000", 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.531e-05 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.191574+0000", 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018237780000000001 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 39/38 ECSubWrite(tid=11, reqid=client.4477.0:196, at_version=39'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.05973739, 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042772959999999999, 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8800000000000002e-07 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206341+0000", 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2418e-05 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208467+0000", 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021264159999999999 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208477+0000", 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6090000000000007e-06 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208544+0000", 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6804000000000006e-05 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208555+0000", 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0869e-05 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208869+0000", 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031429800000000002 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208897+0000", 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8416999999999999e-05 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208959+0000", 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2201999999999998e-05 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208984+0000", 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4618999999999999e-05 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210585+0000", 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001601056 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 39/38 ECSubWrite(tid=12, reqid=client.4477.0:197, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.208410+0000", 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.057634946, 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042832929999999996, 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208410+0000", 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208410+0000", 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208411+0000", 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7100000000000011e-07 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208438+0000", 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6424e-05 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209653+0000", 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001215617 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209664+0000", 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0244999999999999e-05 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209722+0000", 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8675000000000003e-05 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209732+0000", 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3470000000000005e-06 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209848+0000", 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011628099999999999 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209861+0000", 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.359e-05 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209921+0000", 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9296000000000003e-05 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210211+0000", 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029074999999999999 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212694+0000", 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024820969999999999 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 39/38 ECSubWrite(tid=13, reqid=client.4477.0:198, at_version=39'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.056470715, 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032744699999999998, 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.305 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2999999999999999e-07 2024-09-15T01:32:51.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209623+0000", 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7089999999999998e-05 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211317+0000", 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016948569999999999 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211325+0000", 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9699999999999999e-06 2024-09-15T01:32:51.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211380+0000", 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4938000000000002e-05 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211389+0000", 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7439999999999996e-06 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211510+0000", 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012121 2024-09-15T01:32:51.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211545+0000", 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5209999999999997e-05 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211601+0000", 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5541000000000002e-05 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211625+0000", 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4334000000000001e-05 2024-09-15T01:32:51.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212849+0000", 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012236459999999999 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 39/31 ECSubWrite(tid=58, reqid=client.4477.0:240, at_version=39'20, trim_to=0'0, roll_forward_to=39'18))", 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.279710+0000", 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9863352170000006, 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029601300000000001, 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.279710+0000", 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.279710+0000", 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.279711+0000", 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.218e-06 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280721+0000", 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010096510000000001 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280766+0000", 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4734999999999998e-05 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280779+0000", 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3576e-05 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280836+0000", 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6864999999999999e-05 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280849+0000", 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3019e-05 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281118+0000", 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026896899999999998 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281128+0000", 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4450000000000002e-06 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281243+0000", 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115011 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281261+0000", 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8712999999999999e-05 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.282670+0000", 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001408928 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 39/38 ECSubWrite(tid=46, reqid=client.4477.0:247, at_version=39'9, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.298188+0000", 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9678570109999995, 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025692179999999998, 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298188+0000", 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298188+0000", 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298189+0000", 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.172e-06 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298425+0000", 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023576700000000001 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298454+0000", 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8711999999999999e-05 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298462+0000", 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4649999999999994e-06 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298576+0000", 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113979 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298588+0000", 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1473e-05 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.299079+0000", 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049119500000000004 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.299089+0000", 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7229999999999999e-06 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.299215+0000", 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012652400000000001 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.299246+0000", 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0772000000000003e-05 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.300757+0000", 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015114359999999999 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.241466+0000", 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0245788090000003, 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080416059999999998, 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241466+0000", 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241466+0000", 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241467+0000", 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3999999999999998e-07 2024-09-15T01:32:51.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241497+0000", 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0596999999999999e-05 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243396+0000", 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018991819999999999 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243404+0000", 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2119999999999997e-06 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243529+0000", 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012554800000000001 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243541+0000", 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1834000000000001e-05 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243948+0000", 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040691300000000001 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244199+0000", 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025096899999999998 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244271+0000", 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1666000000000001e-05 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245015+0000", 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00074422900000000001 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249508+0000", 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044932160000000004 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0235572519999998, 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056362690000000002, 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0699999999999998e-07 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242511+0000", 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2926999999999999e-05 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242662+0000", 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015114499999999999 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242671+0000", 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7730000000000005e-06 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242736+0000", 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4651000000000002e-05 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242746+0000", 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0203e-05 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242974+0000", 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022804800000000001 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243002+0000", 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7792000000000001e-05 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243044+0000", 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2657999999999997e-05 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243365+0000", 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032055899999999999 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248124+0000", 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047592060000000002 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242514+0000", 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0235309880000001, 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090428599999999998, 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242514+0000", 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242514+0000", 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242515+0000", 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242548+0000", 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3615000000000003e-05 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242692+0000", 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014390900000000001 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242698+0000", 2024-09-15T01:32:51.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4160000000000003e-06 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243566+0000", 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086795700000000004 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243574+0000", 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6889999999999993e-06 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243798+0000", 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000223611 2024-09-15T01:32:51.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244198+0000", 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040037200000000002 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244270+0000", 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1956e-05 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244333+0000", 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2889e-05 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251557+0000", 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072240569999999999 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 44/38 ECSubWrite(tid=20, reqid=osd.0.0:20, at_version=44'12, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.023479129, 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052822889999999999, 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4600000000000001e-07 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242585+0000", 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9040000000000001e-05 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243433+0000", 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084765999999999997 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243441+0000", 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4179999999999997e-06 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243507+0000", 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5222999999999995e-05 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243517+0000", 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0567000000000001e-05 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243744+0000", 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022733200000000001 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244198+0000", 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045302899999999998 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244269+0000", 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1445999999999993e-05 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244295+0000", 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5799000000000002e-05 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247848+0000", 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003553629 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243616+0000", 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0224288210000001, 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081799560000000004, 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243616+0000", 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243616+0000", 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243617+0000", 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243646+0000", 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.879e-05 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243680+0000", 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4082000000000003e-05 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243686+0000", 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1589999999999998e-06 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243983+0000", 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029721700000000001 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243991+0000", 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6240000000000002e-06 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244172+0000", 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018103300000000001 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244199+0000", 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7254999999999999e-05 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244271+0000", 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.203e-05 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245044+0000", 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077298299999999998 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251796+0000", 2024-09-15T01:32:51.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006752096 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.247492+0000", 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0185534120000002, 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045361799999999999, 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247492+0000", 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247492+0000", 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247494+0000", 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7230000000000001e-06 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247507+0000", 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3421000000000001e-05 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247558+0000", 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0801999999999999e-05 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247567+0000", 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9889999999999998e-06 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247635+0000", 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8623999999999999e-05 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247647+0000", 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1844999999999999e-05 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247812+0000", 2024-09-15T01:32:51.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016474600000000001 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247865+0000", 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3183000000000002e-05 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247909+0000", 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3645e-05 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247928+0000", 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9049e-05 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252028+0000", 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041001529999999996 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 44/38 ECSubWrite(tid=21, reqid=osd.0.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0178194559999998, 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040969320000000002, 2024-09-15T01:32:51.350 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:32:51.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248235+0000", 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7180000000000002e-06 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248464+0000", 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022932900000000001 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248473+0000", 2024-09-15T01:32:51.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.789e-06 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248516+0000", 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3380999999999997e-05 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248526+0000", 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2969999999999992e-06 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248624+0000", 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8672000000000007e-05 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248645+0000", 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0327e-05 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248697+0000", 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2602000000000002e-05 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248930+0000", 2024-09-15T01:32:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023236900000000001 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252323+0000", 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003393063 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 44/38 ECSubWrite(tid=22, reqid=client.4477.0:293, at_version=44'14, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.271248+0000", 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9947971600000001, 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031070799999999999, 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271248+0000", 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271248+0000", 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271250+0000", 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.483e-06 2024-09-15T01:32:51.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271392+0000", 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014198600000000001 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271464+0000", 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2108999999999997e-05 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271473+0000", 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2410000000000001e-06 2024-09-15T01:32:51.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271641+0000", 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000167918 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271657+0000", 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6441000000000001e-05 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271939+0000", 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028217699999999999 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271952+0000", 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2741999999999999e-05 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272039+0000", 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6525000000000003e-05 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272070+0000", 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1443999999999999e-05 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.274355+0000", 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022850140000000001 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 44/31 ECSubWrite(tid=92, reqid=client.4477.0:309, at_version=44'31, trim_to=0'0, roll_forward_to=44'28))", 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.274714+0000", 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9913315569999996, 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023409809999999998, 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.274714+0000", 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.274714+0000", 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.274715+0000", 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1299999999999998e-07 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.276162+0000", 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001447208 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.276206+0000", 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4638000000000002e-05 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.276219+0000", 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.253e-05 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.276319+0000", 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9766000000000007e-05 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.276329+0000", 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9369999999999999e-06 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.276519+0000", 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000190187 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.276565+0000", 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6375000000000002e-05 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.276667+0000", 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101708 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.276776+0000", 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108578 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.277055+0000", 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000279141 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 44/31 ECSubWrite(tid=96, reqid=client.4477.0:321, at_version=44'36, trim_to=0'0, roll_forward_to=44'33))", 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.328792+0000", 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9372528720000002, 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026785049999999999, 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:51.365 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.328792+0000", 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.328792+0000", 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.328793+0000", 2024-09-15T01:32:51.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-06 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329046+0000", 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000252978 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329077+0000", 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0784999999999997e-05 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329083+0000", 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0730000000000001e-06 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329127+0000", 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4230000000000002e-05 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329136+0000", 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6610000000000006e-06 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329438+0000", 2024-09-15T01:32:51.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030188200000000001 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329447+0000", 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7669999999999992e-06 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329543+0000", 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5991999999999998e-05 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329562+0000", 2024-09-15T01:32:51.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8906000000000002e-05 2024-09-15T01:32:51.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:51.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:51.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:51.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.331471+0000", 2024-09-15T01:32:51.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001909191 2024-09-15T01:32:51.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.370 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:51.370 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:51.370 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:51.370 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:32:51.372 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:32:51.372 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:51.370+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:51.473 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:51.471+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:51.480 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:51.480 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:51.480 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:32:51.480 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:51.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:32:51.574 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:51.572+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:51.589 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:51.589 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:51.589 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:32:51.589 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:32:51.589 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:51.597 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:32:51.674 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:51.673+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:51.696 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:32:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:32:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:32:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as2 44/38 ECSubWrite(tid=24, reqid=osd.3.0:24, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.240642+0000", 2024-09-15T01:32:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.452831421, 2024-09-15T01:32:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018810250000000001, 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240642+0000", 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240642+0000", 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240642+0000", 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0599999999999999e-07 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240655+0000", 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2584e-05 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240684+0000", 2024-09-15T01:32:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9045999999999999e-05 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240688+0000", 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9929999999999997e-06 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240721+0000", 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2867999999999998e-05 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240729+0000", 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6489999999999997e-06 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242915+0000", 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021854550000000002 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243256+0000", 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034129400000000002 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243353+0000", 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6564999999999994e-05 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243364+0000", 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1262e-05 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259452+0000", 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016088327999999999 2024-09-15T01:32:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as2 44/38 ECSubWrite(tid=25, reqid=osd.3.0:25, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.242005+0000", 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4514689560000003, 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022660409999999999, 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242005+0000", 2024-09-15T01:32:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242005+0000", 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242005+0000", 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242324+0000", 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031827 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242413+0000", 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9316999999999999e-05 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242420+0000", 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.435e-06 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242951+0000", 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00053097300000000001 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242959+0000", 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8539999999999997e-06 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243124+0000", 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016528699999999999 2024-09-15T01:32:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243257+0000", 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013250299999999999 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243353+0000", 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6482000000000002e-05 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243380+0000", 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6288e-05 2024-09-15T01:32:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264665+0000", 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021285314 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs2 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.242999+0000", 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4504745119999995, 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022065234, 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242999+0000", 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242999+0000", 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242999+0000", 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2999999999999999e-07 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243014+0000", 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5074e-05 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243068+0000", 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3211999999999998e-05 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243076+0000", 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5920000000000001e-06 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243113+0000", 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6418000000000003e-05 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243120+0000", 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8380000000000002e-06 2024-09-15T01:32:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243259+0000", 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013872 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243356+0000", 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6434999999999994e-05 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244107+0000", 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075110800000000001 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244123+0000", 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6833999999999998e-05 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.265064+0000", 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020940772999999999 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 44/31 ECSubWrite(tid=30, reqid=osd.5.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.243886+0000", 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4495879909999996, 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020918115000000001, 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243886+0000", 2024-09-15T01:32:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243886+0000", 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243886+0000", 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3e-07 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243897+0000", 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1097999999999999e-05 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243928+0000", 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0669000000000001e-05 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243934+0000", 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9630000000000003e-06 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243975+0000", 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0602000000000002e-05 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243983+0000", 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2849999999999995e-06 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244045+0000", 2024-09-15T01:32:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2473999999999999e-05 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244109+0000", 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3540000000000005e-05 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244144+0000", 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4715999999999998e-05 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244415+0000", 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027119900000000002 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264804+0000", 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020388838999999999 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 44/31 ECSubWrite(tid=63, reqid=osd.14.0:62, at_version=0'0, trim_to=0'0, roll_forward_to=44'24))", 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.243950+0000", 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.449523363, 2024-09-15T01:32:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021221151000000001, 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243950+0000", 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243950+0000", 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243951+0000", 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243958+0000", 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6890000000000004e-06 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243980+0000", 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1979000000000001e-05 2024-09-15T01:32:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243985+0000", 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1050000000000003e-06 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244044+0000", 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9011000000000001e-05 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244053+0000", 2024-09-15T01:32:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7199999999999995e-06 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244163+0000", 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011003800000000001 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244178+0000", 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4806e-05 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244202+0000", 2024-09-15T01:32:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4312e-05 2024-09-15T01:32:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244347+0000", 2024-09-15T01:32:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000144532 2024-09-15T01:32:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.265171+0000", 2024-09-15T01:32:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020824529000000001 2024-09-15T01:32:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.9s1 44/31 ECSubWrite(tid=64, reqid=osd.14.0:60, at_version=0'0, trim_to=0'0, roll_forward_to=44'18))", 2024-09-15T01:32:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.245084+0000", 2024-09-15T01:32:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4483892770000004, 2024-09-15T01:32:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020157773, 2024-09-15T01:32:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245084+0000", 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245084+0000", 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245085+0000", 2024-09-15T01:32:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6400000000000002e-07 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245094+0000", 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1519999999999998e-06 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245123+0000", 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9331000000000001e-05 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245130+0000", 2024-09-15T01:32:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2210000000000002e-06 2024-09-15T01:32:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245176+0000", 2024-09-15T01:32:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6777999999999998e-05 2024-09-15T01:32:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245185+0000", 2024-09-15T01:32:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0250000000000008e-06 2024-09-15T01:32:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245277+0000", 2024-09-15T01:32:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1860000000000005e-05 2024-09-15T01:32:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245286+0000", 2024-09-15T01:32:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5720000000000003e-06 2024-09-15T01:32:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245316+0000", 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0382000000000002e-05 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245328+0000", 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1381e-05 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.265242+0000", 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019914406999999999 2024-09-15T01:32:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs2 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.247258+0000", 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4462150920000001, 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018265526000000001, 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247258+0000", 2024-09-15T01:32:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247258+0000", 2024-09-15T01:32:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247259+0000", 2024-09-15T01:32:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247269+0000", 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4029999999999996e-06 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247297+0000", 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8841000000000001e-05 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247304+0000", 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819999999999998e-06 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247349+0000", 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5028999999999999e-05 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247360+0000", 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0708e-05 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247486+0000", 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000126434 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247491+0000", 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2650000000000004e-06 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247518+0000", 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6903999999999999e-05 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247527+0000", 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4419999999999998e-06 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.265524+0000", 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017997168000000001 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as2 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.247665+0000", 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4458085220000001, 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018084280000000001, 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247665+0000", 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247665+0000", 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247666+0000", 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8599999999999998e-07 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247673+0000", 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7940000000000003e-06 2024-09-15T01:32:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247695+0000", 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1219000000000001e-05 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247702+0000", 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3270000000000003e-06 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247740+0000", 2024-09-15T01:32:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8415999999999999e-05 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247750+0000", 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247845+0000", 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4376000000000006e-05 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247849+0000", 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.177e-06 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247864+0000", 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5688999999999998e-05 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247871+0000", 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3430000000000001e-06 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.265749+0000", 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017878508000000001 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s2 44/38 ECSubWrite(tid=21, reqid=osd.0.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.248814+0000", 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4446598440000002, 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017093291, 2024-09-15T01:32:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248814+0000", 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248814+0000", 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248814+0000", 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.29e-07 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248819+0000", 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0329999999999999e-06 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248875+0000", 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6626000000000003e-05 2024-09-15T01:32:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248881+0000", 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5369999999999997e-06 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248944+0000", 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2988000000000001e-05 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248950+0000", 2024-09-15T01:32:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5450000000000003e-06 2024-09-15T01:32:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249016+0000", 2024-09-15T01:32:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6206999999999998e-05 2024-09-15T01:32:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250716+0000", 2024-09-15T01:32:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017000069999999999 2024-09-15T01:32:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250782+0000", 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5853999999999997e-05 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250989+0000", 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020694499999999999 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.265907+0000", 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01491842 2024-09-15T01:32:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 44/31 ECSubWrite(tid=65, reqid=osd.14.0:65, at_version=44'26, trim_to=0'0, roll_forward_to=44'24))", 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.249459+0000", 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4440146189999998, 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016594660000000001, 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249459+0000", 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249459+0000", 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249459+0000", 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.75e-07 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249485+0000", 2024-09-15T01:32:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5675999999999999e-05 2024-09-15T01:32:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249506+0000", 2024-09-15T01:32:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.126e-05 2024-09-15T01:32:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249510+0000", 2024-09-15T01:32:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7299999999999999e-06 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249538+0000", 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7470999999999998e-05 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249546+0000", 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6489999999999997e-06 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250969+0000", 2024-09-15T01:32:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014230110000000001 2024-09-15T01:32:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251007+0000", 2024-09-15T01:32:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7552999999999998e-05 2024-09-15T01:32:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251122+0000", 2024-09-15T01:32:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114968 2024-09-15T01:32:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251141+0000", 2024-09-15T01:32:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8926000000000001e-05 2024-09-15T01:32:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.266054+0000", 2024-09-15T01:32:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014912941000000001 2024-09-15T01:32:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 44/31 ECSubWrite(tid=28, reqid=osd.8.0:28, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:32:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.250028+0000", 2024-09-15T01:32:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.443445047, 2024-09-15T01:32:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023067219, 2024-09-15T01:32:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250028+0000", 2024-09-15T01:32:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250028+0000", 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250029+0000", 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.67e-07 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250046+0000", 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7005000000000001e-05 2024-09-15T01:32:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250068+0000", 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2430999999999999e-05 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250072+0000", 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4149999999999999e-06 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250119+0000", 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7848e-05 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250130+0000", 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0742e-05 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250986+0000", 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085551500000000005 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251007+0000", 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1841999999999999e-05 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251122+0000", 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114836 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251166+0000", 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3868000000000001e-05 2024-09-15T01:32:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273096+0000", 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021929549999999999 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.251582+0000", 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4418913509999998, 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024275702, 2024-09-15T01:32:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251582+0000", 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251582+0000", 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251583+0000", 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5400000000000001e-07 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251611+0000", 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8152000000000001e-05 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251641+0000", 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9611000000000002e-05 2024-09-15T01:32:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251650+0000", 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3170000000000008e-06 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251681+0000", 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1449000000000003e-05 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251687+0000", 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1140000000000002e-06 2024-09-15T01:32:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.254374+0000", 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026860320000000001 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.254381+0000", 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.413e-06 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256547+0000", 2024-09-15T01:32:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021657069999999998 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256579+0000", 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2326999999999997e-05 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275858+0000", 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019278925999999998 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 44/31 ECSubWrite(tid=30, reqid=osd.8.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.257425+0000", 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4360489669999996, 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018668435000000001, 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257425+0000", 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257425+0000", 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257425+0000", 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257435+0000", 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3149999999999998e-06 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257458+0000", 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3363e-05 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257462+0000", 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9960000000000004e-06 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257503+0000", 2024-09-15T01:32:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1474000000000003e-05 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257508+0000", 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9069999999999998e-06 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257573+0000", 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4609000000000004e-05 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257578+0000", 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5550000000000002e-06 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259196+0000", 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001617159 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259225+0000", 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9567999999999999e-05 2024-09-15T01:32:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.276093+0000", 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016867785999999999 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.270894+0000", 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4225792999999998, 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016536773000000001, 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270894+0000", 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270894+0000", 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270895+0000", 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4000000000000001e-07 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270911+0000", 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6050999999999999e-05 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270944+0000", 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2478999999999999e-05 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270950+0000", 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.669e-06 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270992+0000", 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1894999999999998e-05 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271010+0000", 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7623000000000001e-05 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271095+0000", 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5515999999999999e-05 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272837+0000", 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017415589999999999 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275485+0000", 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026481550000000001 2024-09-15T01:32:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275503+0000", 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8428e-05 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.287431+0000", 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011927658000000001 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s2 44/38 ECSubWrite(tid=22, reqid=client.4477.0:293, at_version=44'14, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.271033+0000", 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.422440742, 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015139394, 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271033+0000", 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271033+0000", 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271033+0000", 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6000000000000003e-07 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271236+0000", 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271269+0000", 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2851000000000001e-05 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271276+0000", 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2389999999999996e-06 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271317+0000", 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1062000000000001e-05 2024-09-15T01:32:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271326+0000", 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8789999999999992e-06 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271531+0000", 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000204377 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272837+0000", 2024-09-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001306478 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275485+0000", 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026479120000000001 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275527+0000", 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2113000000000002e-05 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.286172+0000", 2024-09-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010644941999999999 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 44/31 ECSubWrite(tid=34, reqid=client.4477.0:296, at_version=44'25, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.271419+0000", 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4220545680000001, 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016338892000000001, 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271419+0000", 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271419+0000", 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271420+0000", 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.68e-07 2024-09-15T01:32:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271435+0000", 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5e-05 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271468+0000", 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3432000000000001e-05 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271473+0000", 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4480000000000002e-06 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271528+0000", 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4490999999999997e-05 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271536+0000", 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2640000000000008e-06 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271620+0000", 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3562999999999999e-05 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272838+0000", 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012179389999999999 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275487+0000", 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026495999999999998 2024-09-15T01:32:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275545+0000", 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7221e-05 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.287758+0000", 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012213366 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.272352+0000", 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4211211800000001, 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015840379000000002, 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272352+0000", 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272352+0000", 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272353+0000", 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272365+0000", 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2207e-05 2024-09-15T01:32:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272387+0000", 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2039e-05 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272393+0000", 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5419999999999996e-06 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272432+0000", 2024-09-15T01:32:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8897999999999999e-05 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272442+0000", 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0052e-05 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272515+0000", 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3467000000000007e-05 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272838+0000", 2024-09-15T01:32:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032300900000000002 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275489+0000", 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026506490000000001 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275530+0000", 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1037e-05 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.288193+0000", 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012662995 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 44/38 ECSubWrite(tid=62, reqid=client.4477.0:308, at_version=44'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.273432+0000", 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4200418289999996, 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014838616000000001, 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273432+0000", 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273432+0000", 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273432+0000", 2024-09-15T01:32:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4199999999999995e-07 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273447+0000", 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4948e-05 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273483+0000", 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5670000000000002e-05 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273490+0000", 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7619999999999998e-06 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273518+0000", 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7920999999999999e-05 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273523+0000", 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9259999999999999e-06 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273609+0000", 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6703e-05 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275493+0000", 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001884204 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275530+0000", 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6251999999999998e-05 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275541+0000", 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1080000000000001e-05 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.288270+0000", 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012729508 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 47/31 ECSubWrite(tid=51, reqid=client.4477.0:371, at_version=47'39, trim_to=0'0, roll_forward_to=47'36))", 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:49.359666+0000", 2024-09-15T01:32:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.3338075570000001, 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.65040829099999997, 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.359666+0000", 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.359666+0000", 2024-09-15T01:32:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.359667+0000", 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.92e-07 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.359695+0000", 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7982000000000001e-05 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009537+0000", 2024-09-15T01:32:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.64984255099999999 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009542+0000", 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999998e-06 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009587+0000", 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5257999999999999e-05 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009594+0000", 2024-09-15T01:32:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9859999999999997e-06 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009679+0000", 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4478e-05 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009685+0000", 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9349999999999999e-06 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009721+0000", 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6588000000000003e-05 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009742+0000", 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0766000000000001e-05 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.010074+0000", 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033208499999999998 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s2 47/38 ECSubWrite(tid=32, reqid=client.4477.0:386, at_version=47'21, trim_to=0'0, roll_forward_to=45'16))", 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:49.387720+0000", 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.3057534569999998, 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.89050907300000004, 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.387720+0000", 2024-09-15T01:32:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.387720+0000", 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.387721+0000", 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:32:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.387737+0000", 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6359999999999999e-05 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277497+0000", 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.88975975900000004 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277502+0000", 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6010000000000002e-06 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:32:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277615+0000", 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112551 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277628+0000", 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2486e-05 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277734+0000", 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106314 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277769+0000", 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5200999999999997e-05 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277815+0000", 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5788999999999998e-05 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277835+0000", 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.014e-05 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.278229+0000", 2024-09-15T01:32:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039418500000000002 2024-09-15T01:32:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:32:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:32:51.808 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:51.810 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:32:51.810 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:51.808+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:51.884 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:32:51.910 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:51.908+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:51.947 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:51.947 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:51.947 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:32:51.947 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:51.955 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:32:52.010 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:52.009+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:52.011 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:32:52.011 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:32:52.031 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:32:52.031 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:32:52.051 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:32:52.051 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:32:52.052 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:32:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:32:52.053 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:52.061 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:32:52.070 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.10 to 1 (10000) 2024-09-15T01:32:52.070 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-15T01:32:52.071 INFO:tasks.rados.rados.0.smithi175.stdout:153: done (0 left) 2024-09-15T01:32:52.071 INFO:tasks.rados.rados.0.smithi175.stdout:154: read oid 34 snap 7 2024-09-15T01:32:52.071 INFO:tasks.rados.rados.0.smithi175.stdout:154: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-15T01:32:52.071 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 45 current snap is 9 2024-09-15T01:32:52.072 INFO:tasks.rados.rados.0.smithi175.stdout:155: seq_num 93 ranges {2244608=802816,3047424=638976,3686400=131072} 2024-09-15T01:32:52.072 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:32:52.072 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:32:52.089 INFO:tasks.rados.rados.0.smithi175.stdout:155: writing smithi17529310-45 from 2244608 to 3047424 tid 1 2024-09-15T01:32:52.092 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-15T01:32:52.093 INFO:tasks.rados.rados.0.smithi175.stdout:155: writing smithi17529310-45 from 3047424 to 3686400 tid 2 2024-09-15T01:32:52.095 INFO:tasks.rados.rados.0.smithi175.stdout:155: writing smithi17529310-45 from 3686400 to 3817472 tid 3 2024-09-15T01:32:52.095 INFO:tasks.rados.rados.0.smithi175.stdout:156: snap_remove snap 6 2024-09-15T01:32:52.101 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:32:52.101 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:32:52.111 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:52.110+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:52.121 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:32:52.122 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:32:52.141 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:32:52.141 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:32:52.161 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:32:52.161 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:32:52.166 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:52.166 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:52.174 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:32:52.179 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:32:52.180 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:32:52.198 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:32:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:32:52.212 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:52.211+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:52.220 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:32:52.220 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:32:52.251 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:32:52.251 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:32:52.270 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:32:52.270 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:32:52.272 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:52.272 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:52.280 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:32:52.289 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:32:52.289 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:32:52.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:32:52.292+0000 7f5402ffd640 0 -- 172.21.15.42:0/210430795 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7f5404194870 msgr2=0x7f5404196c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-15T01:32:52.324 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:52.323+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:52.377 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:52.377 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:52.386 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:32:52.427 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:52.426+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:52.515 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:52.513+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:52.524 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:52.524 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:52.533 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-15T01:32:52.616 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:52.614+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:52.629 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:52.630 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:52.638 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-15T01:32:52.717 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:52.715+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:52.734 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:52.734 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:52.743 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-15T01:32:52.818 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:52.816+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:52.839 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:52.839 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:52.847 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:32:52.919 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:52.918+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:52.958 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:52.959 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:52.967 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:32:53.020 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:53.018+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:53.066 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:53.066 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:53.073 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.11 to 1 (10000) 2024-09-15T01:32:53.073 INFO:tasks.rados.rados.0.smithi175.stdout:156: done (2 left) 2024-09-15T01:32:53.074 INFO:tasks.rados.rados.0.smithi175.stdout:157: read oid 30 snap -1 2024-09-15T01:32:53.074 INFO:tasks.rados.rados.0.smithi175.stdout:157: expect (ObjNum 89 snap 7 seq_num 89) 2024-09-15T01:32:53.074 INFO:tasks.rados.rados.0.smithi175.stdout:158: snap_remove snap 8 2024-09-15T01:32:53.075 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:32:53.102 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-09-15T01:32:53.160 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:53.157+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:53.212 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:53.212 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:53.221 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:32:53.221 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:53.219+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:53.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:32:53.228 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:32:53.249 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:32:53.250 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:32:53.321 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:53.320+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:53.359 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:53.359 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:53.367 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:32:53.423 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:53.420+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:53.465 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:53.465 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:53.473 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:32:53.558 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:53.556+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:53.611 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:53.612 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:53.620 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:32:53.623 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:53.622+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:53.724 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:53.723+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:53.756 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:53.757 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:53.765 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:32:53.825 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:53.824+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:53.861 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:53.862 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:53.869 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:32:53.926 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:53.925+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:53.967 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:53.967 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:53.976 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-15T01:32:54.027 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:54.025+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:32:54.073 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:54.073 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:54.077 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.12 to 1 (10000) 2024-09-15T01:32:54.077 INFO:tasks.rados.rados.0.smithi175.stdout:158: done (3 left) 2024-09-15T01:32:54.078 INFO:tasks.rados.rados.0.smithi175.stdout:159: setattr oid 3 current snap is 9 2024-09-15T01:32:54.078 INFO:tasks.rados.rados.0.smithi175.stdout:154: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-15T01:32:54.085 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-15T01:32:54.095 INFO:tasks.rados.rados.0.smithi175.stdout:155: finishing write tid 1 to smithi17529310-45 2024-09-15T01:32:54.095 INFO:tasks.rados.rados.0.smithi175.stdout:155: finishing write tid 2 to smithi17529310-45 2024-09-15T01:32:54.095 INFO:tasks.rados.rados.0.smithi175.stdout:155: finishing write tid 3 to smithi17529310-45 2024-09-15T01:32:54.096 INFO:tasks.rados.rados.0.smithi175.stdout:155: finishing write tid 4 to smithi17529310-45 2024-09-15T01:32:54.096 INFO:tasks.rados.rados.0.smithi175.stdout:155: finishing write tid 5 to smithi17529310-45 2024-09-15T01:32:54.096 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 45 v 23 (ObjNum 93 snap 9 seq_num 93) dirty exists 2024-09-15T01:32:54.096 INFO:tasks.rados.rados.0.smithi175.stdout:155: left oid 45 (ObjNum 93 snap 9 seq_num 93) 2024-09-15T01:32:54.096 INFO:tasks.rados.rados.0.smithi175.stdout:157: expect (ObjNum 89 snap 7 seq_num 89) 2024-09-15T01:32:54.096 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:32:54.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-09-15T01:32:54.140 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:54.138+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:54.157 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 3 v 32 (ObjNum 92 snap 8 seq_num 92) dirty exists 2024-09-15T01:32:54.157 INFO:tasks.rados.rados.0.smithi175.stdout:154: done (3 left) 2024-09-15T01:32:54.157 INFO:tasks.rados.rados.0.smithi175.stdout:155: done (2 left) 2024-09-15T01:32:54.157 INFO:tasks.rados.rados.0.smithi175.stdout:157: done (1 left) 2024-09-15T01:32:54.158 INFO:tasks.rados.rados.0.smithi175.stdout:159: done (0 left) 2024-09-15T01:32:54.158 INFO:tasks.rados.rados.0.smithi175.stdout:160: read oid 6 snap -1 2024-09-15T01:32:54.158 INFO:tasks.rados.rados.0.smithi175.stdout:160: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-15T01:32:54.158 INFO:tasks.rados.rados.0.smithi175.stdout:161: rollback oid 2 current snap is 9 2024-09-15T01:32:54.158 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 2 to 7 2024-09-15T01:32:54.158 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 35 current snap is 9 2024-09-15T01:32:54.158 INFO:tasks.rados.rados.0.smithi175.stdout:162: seq_num 94 ranges {573440=425984,999424=786432,1785856=376832} 2024-09-15T01:32:54.163 INFO:tasks.rados.rados.0.smithi175.stdout:162: writing smithi17529310-35 from 573440 to 999424 tid 1 2024-09-15T01:32:54.167 INFO:tasks.rados.rados.0.smithi175.stdout:162: writing smithi17529310-35 from 999424 to 1785856 tid 2 2024-09-15T01:32:54.170 INFO:tasks.rados.rados.0.smithi175.stdout:162: writing smithi17529310-35 from 1785856 to 2162688 tid 3 2024-09-15T01:32:54.170 INFO:tasks.rados.rados.0.smithi175.stdout:163: rollback oid 7 current snap is 9 2024-09-15T01:32:54.170 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 7 to 7 2024-09-15T01:32:54.170 INFO:tasks.rados.rados.0.smithi175.stdout:164: setattr oid 29 current snap is 9 2024-09-15T01:32:54.170 INFO:tasks.rados.rados.0.smithi175.stdout:161: finishing rollback tid 0 to smithi17529310-2 2024-09-15T01:32:54.170 INFO:tasks.rados.rados.0.smithi175.stdout:161: finishing rollback tid 1 to smithi17529310-2 2024-09-15T01:32:54.171 INFO:tasks.rados.rados.0.smithi175.stdout:160: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-15T01:32:54.174 INFO:tasks.rados.rados.0.smithi175.stdout:162: finishing write tid 1 to smithi17529310-35 2024-09-15T01:32:54.174 INFO:tasks.rados.rados.0.smithi175.stdout:163: finishing rollback tid 0 to smithi17529310-7 2024-09-15T01:32:54.174 INFO:tasks.rados.rados.0.smithi175.stdout:163: finishing rollback tid 1 to smithi17529310-7 2024-09-15T01:32:54.174 INFO:tasks.rados.rados.0.smithi175.stdout:160: done (4 left) 2024-09-15T01:32:54.174 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 5 current snap is 9 2024-09-15T01:32:54.174 INFO:tasks.rados.rados.0.smithi175.stdout:165: seq_num 95 ranges {81920=442368,524288=540672,1064960=622592} 2024-09-15T01:32:54.177 INFO:tasks.rados.rados.0.smithi175.stdout:165: writing smithi17529310-5 from 81920 to 524288 tid 1 2024-09-15T01:32:54.180 INFO:tasks.rados.rados.0.smithi175.stdout:165: writing smithi17529310-5 from 524288 to 1064960 tid 2 2024-09-15T01:32:54.184 INFO:tasks.rados.rados.0.smithi175.stdout:165: writing smithi17529310-5 from 1064960 to 1687552 tid 3 2024-09-15T01:32:54.184 INFO:tasks.rados.rados.0.smithi175.stdout:166: copy_from oid 50 from oid 14 current snap is 9 2024-09-15T01:32:54.185 INFO:tasks.rados.rados.0.smithi175.stdout:163: finishing rollback tid 2 to smithi17529310-7 2024-09-15T01:32:54.185 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 7 v 32 (ObjNum 70 snap 5 seq_num 70) dirty exists 2024-09-15T01:32:54.185 INFO:tasks.rados.rados.0.smithi175.stdout:161: finishing rollback tid 2 to smithi17529310-2 2024-09-15T01:32:54.185 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 2 v 68 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-15T01:32:54.185 INFO:tasks.rados.rados.0.smithi175.stdout:162: finishing write tid 2 to smithi17529310-35 2024-09-15T01:32:54.185 INFO:tasks.rados.rados.0.smithi175.stdout:162: finishing write tid 3 to smithi17529310-35 2024-09-15T01:32:54.185 INFO:tasks.rados.rados.0.smithi175.stdout:161: done (5 left) 2024-09-15T01:32:54.185 INFO:tasks.rados.rados.0.smithi175.stdout:163: done (4 left) 2024-09-15T01:32:54.185 INFO:tasks.rados.rados.0.smithi175.stdout:167: read oid 16 snap -1 2024-09-15T01:32:54.185 INFO:tasks.rados.rados.0.smithi175.stdout:167: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-15T01:32:54.185 INFO:tasks.rados.rados.0.smithi175.stdout:168: snap_remove snap 7 2024-09-15T01:32:54.226 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:54.226 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:54.228 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:54.227+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:54.235 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-15T01:32:54.329 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:54.328+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:54.332 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:54.332 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:54.341 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:32:54.430 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:54.429+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:54.449 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:32:54.531 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:54.529+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:54.546 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:54.555 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:32:54.632 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:54.630+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:54.652 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:54.652 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:54.660 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:32:54.732 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:54.731+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:54.757 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:54.757 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:54.766 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:32:54.833 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:54.832+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:54.904 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:54.904 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:54.912 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:32:54.957 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:54.955+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:55.008 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:55.008 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:55.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:32:55.035 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:55.033+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:55.080 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.13 to 1 (10000) 2024-09-15T01:32:55.080 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-15T01:32:55.082 INFO:tasks.rados.rados.0.smithi175.stdout:168: done (5 left) 2024-09-15T01:32:55.082 INFO:tasks.rados.rados.0.smithi175.stdout:169: rollback oid 22 current snap is 9 2024-09-15T01:32:55.082 INFO:tasks.rados.rados.0.smithi175.stdout:169: done (5 left) 2024-09-15T01:32:55.083 INFO:tasks.rados.rados.0.smithi175.stdout:170: delete oid 45 current snap is 9 2024-09-15T01:32:55.083 INFO:tasks.rados.rados.0.smithi175.stdout:166: finishing copy_from racing read to smithi17529310-50 2024-09-15T01:32:55.083 INFO:tasks.rados.rados.0.smithi175.stdout:166: finishing copy_from to smithi17529310-50 2024-09-15T01:32:55.083 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 50 v 23 (ObjNum 76 snap 5 seq_num 76) dirty exists 2024-09-15T01:32:55.083 INFO:tasks.rados.rados.0.smithi175.stdout:167: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-15T01:32:55.091 INFO:tasks.rados.rados.0.smithi175.stdout:162: finishing write tid 4 to smithi17529310-35 2024-09-15T01:32:55.091 INFO:tasks.rados.rados.0.smithi175.stdout:162: finishing write tid 5 to smithi17529310-35 2024-09-15T01:32:55.091 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 35 v 24 (ObjNum 94 snap 9 seq_num 94) dirty exists 2024-09-15T01:32:55.091 INFO:tasks.rados.rados.0.smithi175.stdout:162: left oid 35 (ObjNum 94 snap 9 seq_num 94) 2024-09-15T01:32:55.092 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 29 v 70 (ObjNum 68 snap 3 seq_num 68) dirty exists 2024-09-15T01:32:55.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-09-15T01:32:55.145 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:55.143+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:55.158 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:55.158 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:55.166 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:32:55.198 INFO:tasks.rados.rados.0.smithi175.stdout:162: done (5 left) 2024-09-15T01:32:55.198 INFO:tasks.rados.rados.0.smithi175.stdout:164: done (4 left) 2024-09-15T01:32:55.198 INFO:tasks.rados.rados.0.smithi175.stdout:166: done (3 left) 2024-09-15T01:32:55.198 INFO:tasks.rados.rados.0.smithi175.stdout:167: done (2 left) 2024-09-15T01:32:55.198 INFO:tasks.rados.rados.0.smithi175.stdout:170: done (1 left) 2024-09-15T01:32:55.199 INFO:tasks.rados.rados.0.smithi175.stdout:171: snap_create 2024-09-15T01:32:55.201 INFO:tasks.rados.rados.0.smithi175.stdout:165: finishing write tid 1 to smithi17529310-5 2024-09-15T01:32:55.202 INFO:tasks.rados.rados.0.smithi175.stdout:165: finishing write tid 2 to smithi17529310-5 2024-09-15T01:32:55.204 INFO:tasks.rados.rados.0.smithi175.stdout:165: finishing write tid 3 to smithi17529310-5 2024-09-15T01:32:55.204 INFO:tasks.rados.rados.0.smithi175.stdout:165: finishing write tid 4 to smithi17529310-5 2024-09-15T01:32:55.236 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:55.235+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:55.262 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:32:55.296 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:55.296 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:55.305 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:32:55.337 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:55.335+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:55.392 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:32:55.392 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:32:55.411 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:32:55.412 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:32:55.430 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:32:55.431 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:32:55.438 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:55.437+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:55.442 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:55.442 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:55.450 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:32:55.450 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:32:55.451 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-15T01:32:55.468 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:32:55.468 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:32:55.487 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:32:55.488 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:32:55.507 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:32:55.507 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:32:55.526 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:32:55.526 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:32:55.539 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:55.537+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:55.544 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:32:55.544 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:32:55.548 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:55.548 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:55.556 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-15T01:32:55.564 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:32:55.564 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:32:55.583 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:32:55.583 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:32:55.602 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:32:55.602 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:32:55.620 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:32:55.620 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:32:55.639 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:55.638+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:55.640 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:32:55.640 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:32:55.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:32:55.639+0000 7f8565d9b640 0 -- 172.21.15.42:0/1932795053 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7f8568192960 msgr2=0x7f8568194dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-15T01:32:55.693 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:55.693 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:55.701 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-15T01:32:55.740 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:55.739+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:55.800 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:55.800 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:55.808 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:32:55.841 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:55.839+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:55.860 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:32:55.861 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:32:55.880 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:32:55.880 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:32:55.912 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:32:55.913 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:55.921 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:32:55.941 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:32:55.940+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:32:56.042 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:56.041+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:56.062 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:56.062 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:56.062 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:32:56.062 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:32:56.062 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:56.067 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.14 to 1 (10000) 2024-09-15T01:32:56.072 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:32:56.073 INFO:tasks.rados.rados.0.smithi175.stdout:171: done (1 left) 2024-09-15T01:32:56.073 INFO:tasks.rados.rados.0.smithi175.stdout:172: rmattr oid 30 current snap is 10 2024-09-15T01:32:56.073 INFO:tasks.rados.rados.0.smithi175.stdout:173: delete oid 7 current snap is 10 2024-09-15T01:32:56.080 INFO:tasks.rados.rados.0.smithi175.stdout:173: done (2 left) 2024-09-15T01:32:56.080 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 12 current snap is 10 2024-09-15T01:32:56.080 INFO:tasks.rados.rados.0.smithi175.stdout:174: seq_num 98 ranges {196608=491520,688128=589824,1277952=557056,1835008=16384} 2024-09-15T01:32:56.084 INFO:tasks.rados.rados.0.smithi175.stdout:174: writing smithi17529310-12 from 196608 to 688128 tid 1 2024-09-15T01:32:56.086 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-09-15T01:32:56.087 INFO:tasks.rados.rados.0.smithi175.stdout:174: writing smithi17529310-12 from 688128 to 1277952 tid 2 2024-09-15T01:32:56.091 INFO:tasks.rados.rados.0.smithi175.stdout:174: writing smithi17529310-12 from 1277952 to 1835008 tid 3 2024-09-15T01:32:56.091 INFO:tasks.rados.rados.0.smithi175.stdout:174: writing smithi17529310-12 from 1835008 to 1851392 tid 4 2024-09-15T01:32:56.091 INFO:tasks.rados.rados.0.smithi175.stdout:175: read oid 25 snap 9 2024-09-15T01:32:56.091 INFO:tasks.rados.rados.0.smithi175.stdout:175: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-15T01:32:56.091 INFO:tasks.rados.rados.0.smithi175.stdout:176: delete oid 9 current snap is 10 2024-09-15T01:32:56.091 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 30 v 27 (ObjNum 89 snap 7 seq_num 89) dirty exists 2024-09-15T01:32:56.091 INFO:tasks.rados.rados.0.smithi175.stdout:174: finishing write tid 1 to smithi17529310-12 2024-09-15T01:32:56.094 INFO:tasks.rados.rados.0.smithi175.stdout:174: finishing write tid 2 to smithi17529310-12 2024-09-15T01:32:56.096 INFO:tasks.rados.rados.0.smithi175.stdout:172: done (4 left) 2024-09-15T01:32:56.097 INFO:tasks.rados.rados.0.smithi175.stdout:176: done (3 left) 2024-09-15T01:32:56.097 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 35 current snap is 10 2024-09-15T01:32:56.097 INFO:tasks.rados.rados.0.smithi175.stdout:177: seq_num 100 ranges {2162688=524288,2686976=753664,3440640=409600} 2024-09-15T01:32:56.111 INFO:tasks.rados.rados.0.smithi175.stdout:177: writing smithi17529310-35 from 2162688 to 2686976 tid 1 2024-09-15T01:32:56.115 INFO:tasks.rados.rados.0.smithi175.stdout:177: writing smithi17529310-35 from 2686976 to 3440640 tid 2 2024-09-15T01:32:56.119 INFO:tasks.rados.rados.0.smithi175.stdout:177: writing smithi17529310-35 from 3440640 to 3850240 tid 3 2024-09-15T01:32:56.119 INFO:tasks.rados.rados.0.smithi175.stdout:178: rollback oid 17 current snap is 10 2024-09-15T01:32:56.119 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 17 to 9 2024-09-15T01:32:56.119 INFO:tasks.rados.rados.0.smithi175.stdout:174: finishing write tid 3 to smithi17529310-12 2024-09-15T01:32:56.119 INFO:tasks.rados.rados.0.smithi175.stdout:174: finishing write tid 4 to smithi17529310-12 2024-09-15T01:32:56.119 INFO:tasks.rados.rados.0.smithi175.stdout:175: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-15T01:32:56.131 INFO:tasks.rados.rados.0.smithi175.stdout:177: finishing write tid 1 to smithi17529310-35 2024-09-15T01:32:56.131 INFO:tasks.rados.rados.0.smithi175.stdout:177: finishing write tid 2 to smithi17529310-35 2024-09-15T01:32:56.131 INFO:tasks.rados.rados.0.smithi175.stdout:177: finishing write tid 3 to smithi17529310-35 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:177: finishing write tid 4 to smithi17529310-35 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:178: finishing rollback tid 0 to smithi17529310-17 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:178: finishing rollback tid 1 to smithi17529310-17 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:177: finishing write tid 5 to smithi17529310-35 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 35 v 31 (ObjNum 100 snap 10 seq_num 100) dirty exists 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:177: left oid 35 (ObjNum 100 snap 10 seq_num 100) 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:178: finishing rollback tid 2 to smithi17529310-17 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 17 v 78 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:175: done (4 left) 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:177: done (3 left) 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:178: done (2 left) 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 28 current snap is 10 2024-09-15T01:32:56.132 INFO:tasks.rados.rados.0.smithi175.stdout:179: seq_num 101 ranges {0=540672,540672=507904,1048576=622592,1671168=32768} 2024-09-15T01:32:56.134 INFO:tasks.rados.rados.0.smithi175.stdout:179: writing smithi17529310-28 from 0 to 540672 tid 1 2024-09-15T01:32:56.137 INFO:tasks.rados.rados.0.smithi175.stdout:179: writing smithi17529310-28 from 540672 to 1048576 tid 2 2024-09-15T01:32:56.141 INFO:tasks.rados.rados.0.smithi175.stdout:179: writing smithi17529310-28 from 1048576 to 1671168 tid 3 2024-09-15T01:32:56.141 INFO:tasks.rados.rados.0.smithi175.stdout:179: writing smithi17529310-28 from 1671168 to 1703936 tid 4 2024-09-15T01:32:56.141 INFO:tasks.rados.rados.0.smithi175.stdout:180: read oid 10 snap -1 2024-09-15T01:32:56.141 INFO:tasks.rados.rados.0.smithi175.stdout:180: expect (ObjNum 58 snap 3 seq_num 58) 2024-09-15T01:32:56.141 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 14 current snap is 10 2024-09-15T01:32:56.141 INFO:tasks.rados.rados.0.smithi175.stdout:179: finishing write tid 1 to smithi17529310-28 2024-09-15T01:32:56.142 INFO:tasks.rados.rados.0.smithi175.stdout:179: finishing write tid 2 to smithi17529310-28 2024-09-15T01:32:56.142 INFO:tasks.rados.rados.0.smithi175.stdout:181: seq_num 102 ranges {2457600=557056,3014656=638976,3653632=524288} 2024-09-15T01:32:56.142 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:56.141+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:56.157 INFO:tasks.rados.rados.0.smithi175.stdout:181: writing smithi17529310-14 from 2457600 to 3014656 tid 1 2024-09-15T01:32:56.161 INFO:tasks.rados.rados.0.smithi175.stdout:181: writing smithi17529310-14 from 3014656 to 3653632 tid 2 2024-09-15T01:32:56.164 INFO:tasks.rados.rados.0.smithi175.stdout:181: writing smithi17529310-14 from 3653632 to 4177920 tid 3 2024-09-15T01:32:56.165 INFO:tasks.rados.rados.0.smithi175.stdout:182: rmattr oid 39 current snap is 10 2024-09-15T01:32:56.165 INFO:tasks.rados.rados.0.smithi175.stdout:182: done (5 left) 2024-09-15T01:32:56.165 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 26 current snap is 10 2024-09-15T01:32:56.165 INFO:tasks.rados.rados.0.smithi175.stdout:179: finishing write tid 3 to smithi17529310-28 2024-09-15T01:32:56.165 INFO:tasks.rados.rados.0.smithi175.stdout:183: seq_num 103 ranges {425984=573440,999424=786432,1785856=376832} 2024-09-15T01:32:56.169 INFO:tasks.rados.rados.0.smithi175.stdout:183: writing smithi17529310-26 from 425984 to 999424 tid 1 2024-09-15T01:32:56.174 INFO:tasks.rados.rados.0.smithi175.stdout:183: writing smithi17529310-26 from 999424 to 1785856 tid 2 2024-09-15T01:32:56.176 INFO:tasks.rados.rados.0.smithi175.stdout:183: writing smithi17529310-26 from 1785856 to 2162688 tid 3 2024-09-15T01:32:56.176 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 33 current snap is 10 2024-09-15T01:32:56.176 INFO:tasks.rados.rados.0.smithi175.stdout:184: seq_num 104 ranges {540672=589824,1130496=540672,1671168=622592} 2024-09-15T01:32:56.182 INFO:tasks.rados.rados.0.smithi175.stdout:184: writing smithi17529310-33 from 540672 to 1130496 tid 1 2024-09-15T01:32:56.186 INFO:tasks.rados.rados.0.smithi175.stdout:184: writing smithi17529310-33 from 1130496 to 1671168 tid 2 2024-09-15T01:32:56.190 INFO:tasks.rados.rados.0.smithi175.stdout:184: writing smithi17529310-33 from 1671168 to 2293760 tid 3 2024-09-15T01:32:56.190 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 19 current snap is 10 2024-09-15T01:32:56.190 INFO:tasks.rados.rados.0.smithi175.stdout:179: finishing write tid 4 to smithi17529310-28 2024-09-15T01:32:56.190 INFO:tasks.rados.rados.0.smithi175.stdout:179: finishing write tid 5 to smithi17529310-28 2024-09-15T01:32:56.190 INFO:tasks.rados.rados.0.smithi175.stdout:185: seq_num 105 ranges {1785856=606208,2392064=688128,3080192=475136} 2024-09-15T01:32:56.202 INFO:tasks.rados.rados.0.smithi175.stdout:185: writing smithi17529310-19 from 1785856 to 2392064 tid 1 2024-09-15T01:32:56.204 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:56.204 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:32:56.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:32:56.204 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:32:56.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.204 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:18 3.6s0 3:6ba0b44a:::smithi17529310-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:56.204 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.181034+0000", 2024-09-15T01:32:56.204 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.019645193999999, 2024-09-15T01:32:56.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085778220000000006, 2024-09-15T01:32:56.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:56.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181034+0000", 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181034+0000", 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181034+0000", 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181038+0000", 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9140000000000001e-06 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181047+0000", 2024-09-15T01:32:56.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6680000000000007e-06 2024-09-15T01:32:56.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181055+0000", 2024-09-15T01:32:56.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0709999999999995e-06 2024-09-15T01:32:56.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.183085+0000", 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020298629999999998 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.183392+0000", 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030756300000000001 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187779+0000", 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043861569999999999 2024-09-15T01:32:56.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187856+0000", 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7914e-05 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.189611+0000", 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754916 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.7s3 34/31 ECSubWrite(tid=17, reqid=client.4477.0:77, at_version=34'6, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.985174934, 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015250150000000001, 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:32:56.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:32:56.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:32:56.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215518+0000", 2024-09-15T01:32:56.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3567000000000001e-05 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230278+0000", 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014760300000000001 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230285+0000", 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.285e-06 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230359+0000", 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4025999999999999e-05 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230367+0000", 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4500000000000004e-06 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230457+0000", 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0457999999999993e-05 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230498+0000", 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0204000000000003e-05 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230554+0000", 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6097999999999997e-05 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230573+0000", 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9252000000000001e-05 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230754+0000", 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018093300000000001 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:90 3.6s0 3:65d6c4b7:::smithi17529310-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.974537015999999, 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.964412163, 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:56.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 90 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:32:56.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226144+0000", 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0490000000000002e-06 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226152+0000", 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5700000000000004e-06 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226157+0000", 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0069999999999997e-06 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.227618+0000", 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014613 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.227698+0000", 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9518000000000007e-05 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.228785+0000", 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010872729999999999 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.228833+0000", 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.85e-05 2024-09-15T01:32:56.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:37.179967+0000", 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.95113379099999995 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:37.179971+0000", 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7620000000000001e-06 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:38.180851+0000", 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0008801570000001 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:38.180855+0000", 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.416e-06 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.187397+0000", 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.006542346 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.187673+0000", 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000275825 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.190554+0000", 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028811650000000002 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.222 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.7s3 34/31 ECSubRead(tid=22, to_read={3:e6e8875f:::smithi17529310-31:head=0,4096,0}, subchunks={3:e6e8875f:::smithi17529310-31:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.967164970999999, 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.074844447999999994, 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4300000000000003e-07 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233521+0000", 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0280000000000003e-06 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308171+0000", 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.074650066000000001 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308178+0000", 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5810000000000002e-06 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308258+0000", 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9977999999999999e-05 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308358+0000", 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100152 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:196 3.6s0 3:65d6c4b7:::smithi17529310-30:head [append 0~0] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.997516534000001, 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007181882, 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.226 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 196 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:32:56.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203168+0000", 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5620000000000003e-06 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203175+0000", 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5599999999999996e-06 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203194+0000", 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8757999999999999e-05 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203201+0000", 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7429999999999996e-06 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203414+0000", 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021300600000000001 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203419+0000", 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1420000000000001e-06 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.204700+0000", 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.001280647 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.204945+0000", 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024550199999999998 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.205894+0000", 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094873799999999997 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206188+0000", 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029375700000000001 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206257+0000", 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9632e-05 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206371+0000", 2024-09-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011411100000000001 2024-09-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206444+0000", 2024-09-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2355999999999996e-05 2024-09-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210281+0000", 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038376740000000001 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210344+0000", 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2694000000000007e-05 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:197 3.6s0 3:65d6c4b7:::smithi17529310-30:head [rollback 3] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.99746294, 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0090962939999999, 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 197 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8299999999999997e-07 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203220+0000", 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0620000000000002e-06 2024-09-15T01:32:56.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203229+0000", 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5329999999999994e-06 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203234+0000", 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000002e-06 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203537+0000", 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030286700000000001 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203539+0000", 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.277e-06 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206748+0000", 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.003209102 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206862+0000", 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011395000000000001 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207832+0000", 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097048299999999996 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207937+0000", 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104583 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207956+0000", 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9381e-05 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208001+0000", 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4642999999999999e-05 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209901+0000", 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001899766 2024-09-15T01:32:56.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211474+0000", 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015733190000000001 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212312+0000", 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083790500000000001 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:198 3.6s0 3:65d6c4b7:::smithi17529310-30:head [append 0~0] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.997178748, 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.009021902, 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 198 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203501+0000", 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4600000000000004e-07 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203505+0000", 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3830000000000002e-06 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203562+0000", 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7120000000000002e-05 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203570+0000", 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:32:56.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203656+0000", 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6273000000000006e-05 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203658+0000", 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4360000000000001e-06 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208296+0000", 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004637574 2024-09-15T01:32:56.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208407+0000", 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110994 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209048+0000", 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064081900000000005 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209141+0000", 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3827999999999997e-05 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209164+0000", 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2201e-05 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209206+0000", 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2021999999999998e-05 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209910+0000", 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070434399999999998 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211939+0000", 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020290159999999998 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212522+0000", 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058283599999999999 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:211 3.6s0 3:6ba0b44a:::smithi17529310-6:head [copy-from ver 0] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.002698220999999, 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.22039597699999999, 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.248 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:32:56.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:32:56.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197982+0000", 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.618e-06 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197988+0000", 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4519999999999996e-06 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198027+0000", 2024-09-15T01:32:56.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9492000000000001e-05 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198036+0000", 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3950000000000007e-06 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198736+0000", 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00069949200000000002 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:56.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198740+0000", 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3470000000000001e-06 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210425+0000", 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011684905000000001 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210622+0000", 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019688099999999999 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418376+0000", 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.20775439500000001 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:212 3.6s0 3:6ba0b44a:::smithi17529310-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.002634572, 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.22052332099999999, 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 212 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:32:56.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198045+0000", 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6499999999999998e-07 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198049+0000", 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4150000000000001e-06 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198058+0000", 2024-09-15T01:32:56.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9830000000000002e-06 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198064+0000", 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6219999999999997e-06 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198782+0000", 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071748199999999997 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198783+0000", 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9400000000000001e-06 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210808+0000", 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012024217 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210861+0000", 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2859999999999999e-05 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418443+0000", 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.20758253099999999 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418509+0000", 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5516000000000001e-05 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418567+0000", 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.889e-05 2024-09-15T01:32:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs6 39/31 ECSubWrite(tid=56, reqid=osd.11.0:56, at_version=39'21, trim_to=0'0, roll_forward_to=38'17))", 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.95788582, 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038235071000000002, 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242794+0000", 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8599999999999997e-07 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242817+0000", 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3655e-05 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280383+0000", 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037566100999999998 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280390+0000", 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280442+0000", 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1650000000000002e-05 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280448+0000", 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7479999999999999e-06 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280660+0000", 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021205900000000001 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280667+0000", 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4710000000000001e-06 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280705+0000", 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7511000000000001e-05 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280722+0000", 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7646999999999999e-05 2024-09-15T01:32:56.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281028+0000", 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030575099999999999 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s1 44/31 ECSubWrite(tid=29, reqid=osd.5.0:29, at_version=44'21, trim_to=0'0, roll_forward_to=40'19))", 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.240231+0000", 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9604479230000003, 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0092854300000000008, 2024-09-15T01:32:56.264 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240231+0000", 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240231+0000", 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240231+0000", 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3200000000000001e-07 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240255+0000", 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3978999999999999e-05 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240354+0000", 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8795000000000005e-05 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240361+0000", 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7109999999999998e-06 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240412+0000", 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1104999999999999e-05 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240422+0000", 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0603e-05 2024-09-15T01:32:56.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241395+0000", 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097287000000000003 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241408+0000", 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.273e-05 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241441+0000", 2024-09-15T01:32:56.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3243e-05 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241457+0000", 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6324000000000002e-05 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249516+0000", 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080588380000000005 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs2 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.958363748, 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011429094000000001, 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242316+0000", 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2900000000000003e-07 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242337+0000", 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1263999999999998e-05 2024-09-15T01:32:56.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242392+0000", 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5408999999999998e-05 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242399+0000", 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.866e-06 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242456+0000", 2024-09-15T01:32:56.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-05 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242466+0000", 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0015e-05 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242685+0000", 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021821399999999999 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242713+0000", 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8589e-05 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242754+0000", 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1173e-05 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242790+0000", 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5167999999999999e-05 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253744+0000", 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010954527 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs1 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9578673640000002, 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010913366000000001, 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.275 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242812+0000", 2024-09-15T01:32:56.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242834+0000", 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1827999999999998e-05 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242901+0000", 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6840000000000004e-05 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242908+0000", 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5440000000000001e-06 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242959+0000", 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1057000000000003e-05 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242969+0000", 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8030000000000008e-06 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243189+0000", 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021987499999999999 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243212+0000", 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2569999999999999e-05 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243258+0000", 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6452000000000002e-05 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243274+0000", 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6426000000000002e-05 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253725+0000", 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010450249 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.280 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs6 44/31 ECSubWrite(tid=82, reqid=client.4477.0:281, at_version=44'36, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9571142819999992, 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011937237, 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243565+0000", 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0500000000000003e-07 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243585+0000", 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0313e-05 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243689+0000", 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103977 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243696+0000", 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0489999999999998e-06 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243791+0000", 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4394000000000005e-05 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243800+0000", 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2439999999999999e-06 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244285+0000", 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000484802 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244313+0000", 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8515000000000001e-05 2024-09-15T01:32:56.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244359+0000", 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5389000000000002e-05 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244375+0000", 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6699000000000001e-05 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.255502+0000", 2024-09-15T01:32:56.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011126250000000001 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s1 44/31 ECSubWrite(tid=30, reqid=osd.5.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243919+0000", 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9567596609999995, 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010421095999999999, 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243919+0000", 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243919+0000", 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243920+0000", 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2399999999999997e-07 2024-09-15T01:32:56.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243929+0000", 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8170000000000005e-06 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245823+0000", 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001894792 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245831+0000", 2024-09-15T01:32:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9160000000000001e-06 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245883+0000", 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789000000000002e-05 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245894+0000", 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0658e-05 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246000+0000", 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105809 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246019+0000", 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250999999999999e-05 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246056+0000", 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6636e-05 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249628+0000", 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035721339999999998 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.254340+0000", 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047124699999999999 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.9s2 44/31 ECSubWrite(tid=64, reqid=osd.14.0:60, at_version=0'0, trim_to=0'0, roll_forward_to=44'18))", 2024-09-15T01:32:56.291 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.245393+0000", 2024-09-15T01:32:56.292 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9552852759999997, 2024-09-15T01:32:56.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087638879999999992, 2024-09-15T01:32:56.292 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.292 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245393+0000", 2024-09-15T01:32:56.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245393+0000", 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245394+0000", 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3900000000000003e-07 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245406+0000", 2024-09-15T01:32:56.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2092e-05 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245442+0000", 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5505e-05 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245454+0000", 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1549000000000001e-05 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245508+0000", 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4781999999999998e-05 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245519+0000", 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0497e-05 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245624+0000", 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105402 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245646+0000", 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1739e-05 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245687+0000", 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0728e-05 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249592+0000", 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039053099999999999 2024-09-15T01:32:56.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.254157+0000", 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045653450000000002 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 44/31 ECSubWrite(tid=36, reqid=osd.7.0:36, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.246398+0000", 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9542802679999998, 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0094625609999999995, 2024-09-15T01:32:56.297 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246398+0000", 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246398+0000", 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246399+0000", 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8e-07 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246422+0000", 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3703e-05 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247337+0000", 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091461400000000001 2024-09-15T01:32:56.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247344+0000", 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1570000000000003e-06 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247394+0000", 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9966000000000002e-05 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247403+0000", 2024-09-15T01:32:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7260000000000008e-06 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247834+0000", 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043114100000000001 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5984000000000001e-05 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247900+0000", 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9941999999999998e-05 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.250749+0000", 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028491440000000001 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.255861+0000", 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051120039999999999 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.302 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.4s4 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.251451+0000", 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9492281790000003, 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085529130000000005, 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251451+0000", 2024-09-15T01:32:56.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251451+0000", 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251451+0000", 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0599999999999999e-07 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251474+0000", 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2266999999999999e-05 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251497+0000", 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.336e-05 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0169999999999999e-06 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251554+0000", 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0251999999999999e-05 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251563+0000", 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8440000000000004e-06 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251810+0000", 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024663300000000001 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251841+0000", 2024-09-15T01:32:56.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0907000000000001e-05 2024-09-15T01:32:56.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251894+0000", 2024-09-15T01:32:56.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3436999999999997e-05 2024-09-15T01:32:56.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251908+0000", 2024-09-15T01:32:56.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4132e-05 2024-09-15T01:32:56.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.260003+0000", 2024-09-15T01:32:56.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080952579999999993 2024-09-15T01:32:56.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.308 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.308 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 44/31 ECSubWrite(tid=39, reqid=client.4477.0:283, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9323922919999994, 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012672954, 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268287+0000", 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.85e-07 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268806+0000", 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00051859699999999998 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272018+0000", 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032124620000000001 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272032+0000", 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4008e-05 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272076+0000", 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3807999999999998e-05 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272085+0000", 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9749999999999996e-06 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272331+0000", 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024564600000000003 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272349+0000", 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7988000000000001e-05 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272412+0000", 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2718000000000005e-05 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272426+0000", 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.280959+0000", 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085329199999999994 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:293 3.6s0 3:65d6c4b7:::smithi17529310-30:head [create,setxattr dgeSiblb`YZWoafOlamThlmYtW[NsgYatRkY]Rkn (17332) in=40b,setxattr c`hS`asrnTY^]RZfhhpk^WaRiegkSORj]tZRqnbU (945) in=40b,setxattr biPi^V_QgOq`^qVaOnjr`odUZpilfdNcYZaf]N]q (594) in=40b,setxattr ilZ^tXpioaqcTlZtZR[RiVbNk[cWS^Q`oYjaj_j[ (17824) in=40b,setxattr _p_ks]sbj_gRlVjZr]OjWXZ]gf (1718) in=26b,setxattr eQ\\\\cjZhQZanRhQZcRjiTbSiomgf]n`[kNbokP^` (4560) in=40b,setxattr gqXgeRc[[^pjV]NraefeRN]mbW\\eSlcSOXdPrmSb (1823) in=40b,setxattr Y]hO^[akiVr`]YhQ[oTnksOSSdslk[h\\idalij_d (701) in=40b,setxattr VqPasiXWWe^eT\\Q`e`__hRmg]O_kYeg_mOrmbaaN (1051) in=40b,setxattr ]]i\\^cfhf_mbRNmfWcpO^Tf[Sa`abbsR_el\\OTgR (1366) in=40b,setxattr e]a_`g\\Of\\ipZOgP]aadXoRp\\q^kh_cpolQ[RjXf (1402) in=40b,setxattr VTaQUZ_lrfT`VmUqPmbhXZciq[kkhXT`nkTblbh` (1298) in=40b,setxattr aUjlO\\kVcZYT^ZU\\jbkfd_VRRa]nqn_[T\\ZhlZp[ (14664) in=40b,setxattr sZScfemtX`pkqf^`dZ]ap (1718) in=21b,setxattr agej`oOViShjhSZSRaYRa_k`gk^ecQqfqgOjpa[q (33) in=40b,setxattr fsad_S`\\kPa[[i[VVsaQOZXpPa]pp\\fOQ`tbY`Qg (13472) in=40b,setxattr c\\g]ORfZrTgatTVmX^dSZtTYaXN^YVoZX[Sajonf (1928) in=40b,setxattr T`^]hVNaYYjRdaNne[XsYdaqR]SSTVTsmUnWXqnp (911) in=40b,setxattr ]rWN_b^TgarYNoqUP]qjm_kNbNiPbpth^PrblPjb (315) in=40b,setxattr XtYoarpQjhPgloXSNSob_agQ\\WY\\\\nThcs[XqPZP (1857) in=40b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.269581+0000", 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9310980660000006, 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088342349999999993, 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 293 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269581+0000", 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269581+0000", 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269583+0000", 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-15T01:32:56.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269783+0000", 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020008499999999999 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269799+0000", 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.611e-05 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269805+0000", 2024-09-15T01:32:56.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1739999999999997e-06 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269863+0000", 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8013000000000002e-05 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269986+0000", 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012216500000000001 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271038+0000", 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001052313 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271289+0000", 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025120200000000001 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271335+0000", 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5512000000000001e-05 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271424+0000", 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9284999999999996e-05 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271758+0000", 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033430299999999999 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.276945+0000", 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051865039999999998 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.278415+0000", 2024-09-15T01:32:56.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001470138 2024-09-15T01:32:56.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.320 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.320 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.320 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:56.324 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:32:56.325 INFO:tasks.rados.rados.0.smithi175.stdout:185: writing smithi17529310-19 from 2392064 to 3080192 tid 2 2024-09-15T01:32:56.325 INFO:tasks.rados.rados.0.smithi175.stdout:185: writing smithi17529310-19 from 3080192 to 3555328 tid 3 2024-09-15T01:32:56.325 INFO:tasks.rados.rados.0.smithi175.stdout:186: snap_create 2024-09-15T01:32:56.325 INFO:tasks.rados.rados.0.smithi175.stdout:179: finishing write tid 6 to smithi17529310-28 2024-09-15T01:32:56.325 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 28 v 39 (ObjNum 101 snap 10 seq_num 101) dirty exists 2024-09-15T01:32:56.325 INFO:tasks.rados.rados.0.smithi175.stdout:179: left oid 28 (ObjNum 101 snap 10 seq_num 101) 2024-09-15T01:32:56.325 INFO:tasks.rados.rados.0.smithi175.stdout:180: expect (ObjNum 58 snap 3 seq_num 58) 2024-09-15T01:32:56.325 INFO:tasks.rados.rados.0.smithi175.stdout:181: finishing write tid 1 to smithi17529310-14 2024-09-15T01:32:56.325 INFO:tasks.rados.rados.0.smithi175.stdout:181: finishing write tid 2 to smithi17529310-14 2024-09-15T01:32:56.325 INFO:tasks.rados.rados.0.smithi175.stdout:181: finishing write tid 3 to smithi17529310-14 2024-09-15T01:32:56.325 INFO:tasks.rados.rados.0.smithi175.stdout:183: finishing write tid 1 to smithi17529310-26 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:183: finishing write tid 2 to smithi17529310-26 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:183: finishing write tid 3 to smithi17529310-26 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:183: finishing write tid 4 to smithi17529310-26 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:183: finishing write tid 5 to smithi17529310-26 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 26 v 36 (ObjNum 103 snap 10 seq_num 103) dirty exists 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:183: left oid 26 (ObjNum 103 snap 10 seq_num 103) 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:184: finishing write tid 1 to smithi17529310-33 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:184: finishing write tid 2 to smithi17529310-33 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:184: finishing write tid 3 to smithi17529310-33 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:184: finishing write tid 4 to smithi17529310-33 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:184: finishing write tid 5 to smithi17529310-33 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 33 v 83 (ObjNum 104 snap 10 seq_num 104) dirty exists 2024-09-15T01:32:56.326 INFO:tasks.rados.rados.0.smithi175.stdout:184: left oid 33 (ObjNum 104 snap 10 seq_num 104) 2024-09-15T01:32:56.327 INFO:tasks.rados.rados.0.smithi175.stdout:165: finishing write tid 5 to smithi17529310-5 2024-09-15T01:32:56.327 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 5 v 8 (ObjNum 95 snap 9 seq_num 95) dirty exists 2024-09-15T01:32:56.327 INFO:tasks.rados.rados.0.smithi175.stdout:165: left oid 5 (ObjNum 95 snap 9 seq_num 95) 2024-09-15T01:32:56.327 INFO:tasks.rados.rados.0.smithi175.stdout:185: finishing write tid 1 to smithi17529310-19 2024-09-15T01:32:56.327 INFO:tasks.rados.rados.0.smithi175.stdout:185: finishing write tid 2 to smithi17529310-19 2024-09-15T01:32:56.327 INFO:tasks.rados.rados.0.smithi175.stdout:185: finishing write tid 3 to smithi17529310-19 2024-09-15T01:32:56.327 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:56.320+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:56.422 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:56.420+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:56.428 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:56.428 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:32:56.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.428 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:500 3.7s0 3:e89d09e8:::smithi17529310-19:head [setxattr _header (57) in=64b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.209208+0000", 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.21832185400000001, 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.21834690300000001, 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 500 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209208+0000", 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209208+0000", 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209209+0000", 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4199999999999995e-07 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209213+0000", 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8410000000000002e-06 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209217+0000", 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.391e-06 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209220+0000", 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6570000000000001e-06 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.210612+0000", 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013926850000000001 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.210830+0000", 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000217655 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211263+0000", 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043276600000000001 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211409+0000", 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014583400000000001 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211418+0000", 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7459999999999995e-06 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211458+0000", 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999999e-05 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211653+0000", 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000195801 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:501 3.7s0 3:e89d09e8:::smithi17529310-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.21830749699999999, 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.218392212, 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "waiting for rw locks", 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.434 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 501 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209225+0000", 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9819999999999998e-06 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209229+0000", 2024-09-15T01:32:56.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3959999999999999e-06 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209233+0000", 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5609999999999999e-06 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211492+0000", 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002259414 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211637+0000", 2024-09-15T01:32:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014431099999999999 2024-09-15T01:32:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:32:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 2 2024-09-15T01:32:56.438 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:56.439 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:32:56.490 INFO:tasks.rados.rados.0.smithi175.stdout:185: finishing write tid 4 to smithi17529310-19 2024-09-15T01:32:56.492 INFO:tasks.rados.rados.0.smithi175.stdout:185: finishing write tid 5 to smithi17529310-19 2024-09-15T01:32:56.492 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 19 v 41 (ObjNum 105 snap 10 seq_num 105) dirty exists 2024-09-15T01:32:56.492 INFO:tasks.rados.rados.0.smithi175.stdout:185: left oid 19 (ObjNum 105 snap 10 seq_num 105) 2024-09-15T01:32:56.523 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:32:56.521+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:32:56.539 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:56.540 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:56.540 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:32:56.540 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:32:56.540 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:56.548 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:32:56.570 INFO:tasks.rados.rados.0.smithi175.stdout:181: finishing write tid 4 to smithi17529310-14 2024-09-15T01:32:56.571 INFO:tasks.rados.rados.0.smithi175.stdout:181: finishing write tid 5 to smithi17529310-14 2024-09-15T01:32:56.571 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 14 v 41 (ObjNum 102 snap 10 seq_num 102) dirty exists 2024-09-15T01:32:56.571 INFO:tasks.rados.rados.0.smithi175.stdout:181: left oid 14 (ObjNum 102 snap 10 seq_num 102) 2024-09-15T01:32:56.624 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:56.623+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:56.692 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:56.692 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:32:56.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:32:56.692 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:32:56.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.692 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:14 3.1s0 3:8930608f:::smithi17529310-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:56.692 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.176805+0000", 2024-09-15T01:32:56.692 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.511944819, 2024-09-15T01:32:56.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020710388, 2024-09-15T01:32:56.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 14 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176805+0000", 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176805+0000", 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176806+0000", 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176810+0000", 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8399999999999997e-06 2024-09-15T01:32:56.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176824+0000", 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4051e-05 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176827+0000", 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5159999999999999e-06 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178153+0000", 2024-09-15T01:32:56.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013253519999999999 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178241+0000", 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8670000000000003e-05 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178708+0000", 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00046673299999999999 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178860+0000", 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015201099999999999 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179098+0000", 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238123 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179138+0000", 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0184e-05 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180215+0000", 2024-09-15T01:32:56.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001076544 2024-09-15T01:32:56.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197335+0000", 2024-09-15T01:32:56.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017120209000000001 2024-09-15T01:32:56.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197516+0000", 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000180657 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:15 3.1s0 3:8930608f:::smithi17529310-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.176858+0000", 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.511891731999999, 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023897231000000001, 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:56.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15 2024-09-15T01:32:56.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.700 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176858+0000", 2024-09-15T01:32:56.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176858+0000", 2024-09-15T01:32:56.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176859+0000", 2024-09-15T01:32:56.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0999999999999999e-07 2024-09-15T01:32:56.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176861+0000", 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4099999999999998e-06 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176875+0000", 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3366e-05 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176882+0000", 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5560000000000002e-06 2024-09-15T01:32:56.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178931+0000", 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002048299 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179014+0000", 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3207000000000005e-05 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197559+0000", 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018545582000000001 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197656+0000", 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7085999999999998e-05 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.200756+0000", 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030992150000000002 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.704 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.705 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:29 3.7s0 3:e2b4e7d9:::smithi17529310-10:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:56.705 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.177051+0000", 2024-09-15T01:32:56.705 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.511698786, 2024-09-15T01:32:56.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020495925000000002, 2024-09-15T01:32:56.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.705 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 29 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177051+0000", 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177051+0000", 2024-09-15T01:32:56.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177052+0000", 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177064+0000", 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1477999999999999e-05 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177072+0000", 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5629999999999992e-06 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177084+0000", 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1557e-05 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178451+0000", 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013673019999999999 2024-09-15T01:32:56.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178548+0000", 2024-09-15T01:32:56.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6639999999999996e-05 2024-09-15T01:32:56.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179029+0000", 2024-09-15T01:32:56.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048111300000000003 2024-09-15T01:32:56.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179171+0000", 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014193800000000001 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2196e-05 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179212+0000", 2024-09-15T01:32:56.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9462999999999999e-05 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179251+0000", 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9155e-05 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197330+0000", 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018078409 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197547+0000", 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021747699999999999 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:30 3.7s0 3:e2b4e7d9:::smithi17529310-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.177099+0000", 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.511651048000001, 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024179603000000001, 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:56.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 30 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177099+0000", 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177099+0000", 2024-09-15T01:32:56.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177100+0000", 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-07 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177103+0000", 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.847e-06 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177111+0000", 2024-09-15T01:32:56.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4780000000000008e-06 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177116+0000", 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999998e-06 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179311+0000", 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002195331 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179405+0000", 2024-09-15T01:32:56.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4611999999999995e-05 2024-09-15T01:32:56.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197589+0000", 2024-09-15T01:32:56.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018183300999999999 2024-09-15T01:32:56.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197665+0000", 2024-09-15T01:32:56.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5878999999999996e-05 2024-09-15T01:32:56.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.201279+0000", 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003614089 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:31 3.1s0 3:8776790e:::smithi17529310-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.177756+0000", 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.510994412999999, 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020342740000000002, 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 31 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177756+0000", 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177756+0000", 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177757+0000", 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178059+0000", 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030214800000000002 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178069+0000", 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0324e-05 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178081+0000", 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1491999999999999e-05 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179041+0000", 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096040399999999999 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179333+0000", 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029163799999999999 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180034+0000", 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070128900000000001 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180155+0000", 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120844 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180166+0000", 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0967e-05 2024-09-15T01:32:56.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180220+0000", 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4048999999999997e-05 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182483+0000", 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022628930000000002 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.198037+0000", 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015553726 2024-09-15T01:32:56.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.198099+0000", 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1824999999999998e-05 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:32 3.1s0 3:8776790e:::smithi17529310-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.178085+0000", 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.510664817999999, 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021154846000000001, 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.723 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178085+0000", 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178085+0000", 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178086+0000", 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0699999999999997e-07 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178088+0000", 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1100000000000001e-06 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178096+0000", 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4419999999999998e-06 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178101+0000", 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1810000000000003e-06 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180259+0000", 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021580839999999998 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180359+0000", 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100028 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181451+0000", 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001092569 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181638+0000", 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018707899999999999 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181720+0000", 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1682999999999997e-05 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182311+0000", 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059133299999999998 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182487+0000", 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017578299999999999 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199166+0000", 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016678622000000001 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199240+0000", 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4425000000000007e-05 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:33 3.1s0 3:8776790e:::smithi17529310-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:32:56.729 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.510645394000001, 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024100798, 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 33 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178108+0000", 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3259999999999998e-06 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178114+0000", 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5209999999999999e-06 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178118+0000", 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7139999999999999e-06 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182355+0000", 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042371070000000004 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:56.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182467+0000", 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011186799999999999 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199900+0000", 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017433015999999999 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199973+0000", 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2897000000000004e-05 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.202206+0000", 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022329049999999999 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:44 3.bs0 3:d6f740a8:::smithi17529310-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.185918+0000", 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.502832347999998, 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.123698503, 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185918+0000", 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185918+0000", 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185919+0000", 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7400000000000003e-07 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185927+0000", 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7860000000000002e-06 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185936+0000", 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7569999999999993e-06 2024-09-15T01:32:56.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185941+0000", 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1889999999999998e-06 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187237+0000", 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001295806 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187337+0000", 2024-09-15T01:32:56.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9630000000000007e-05 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187837+0000", 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000500757 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187919+0000", 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1482000000000004e-05 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187962+0000", 2024-09-15T01:32:56.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2647e-05 2024-09-15T01:32:56.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188001+0000", 2024-09-15T01:32:56.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9070999999999999e-05 2024-09-15T01:32:56.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188078+0000", 2024-09-15T01:32:56.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7713999999999995e-05 2024-09-15T01:32:56.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309538+0000", 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.121459186 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309616+0000", 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8804e-05 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:45 3.bs0 3:d6f740a8:::smithi17529310-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.502804573999999, 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12623172299999999, 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 45 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:32:56.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185948+0000", 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185954+0000", 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1090000000000003e-06 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185958+0000", 2024-09-15T01:32:56.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5159999999999999e-06 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187975+0000", 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002017343 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188054+0000", 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8734999999999997e-05 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309670+0000", 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.121615926 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309756+0000", 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6316999999999994e-05 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.312177+0000", 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002421126 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:77 3.7s0 3:eb933b0c:::smithi17529310-26:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.213552+0000", 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.475197848000001, 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017819044999999999, 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 77 2024-09-15T01:32:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213552+0000", 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213552+0000", 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213553+0000", 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7899999999999997e-07 2024-09-15T01:32:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213558+0000", 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6489999999999999e-06 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213567+0000", 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6130000000000004e-06 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213571+0000", 2024-09-15T01:32:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1740000000000002e-06 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.214963+0000", 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013925350000000001 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215037+0000", 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4158000000000004e-05 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215424+0000", 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038638099999999999 2024-09-15T01:32:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215508+0000", 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3863999999999995e-05 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215521+0000", 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3089e-05 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215553+0000", 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2001000000000001e-05 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215598+0000", 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5510999999999998e-05 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231298+0000", 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015699856000000002 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231371+0000", 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3335000000000002e-05 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:78 3.7s0 3:eb933b0c:::smithi17529310-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.213590+0000", 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.475159999999999, 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020370672999999999, 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 78 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213590+0000", 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213590+0000", 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213591+0000", 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213593+0000", 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3199999999999998e-06 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213600+0000", 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4580000000000004e-06 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213607+0000", 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-06 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215631+0000", 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020248890000000002 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215718+0000", 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6127999999999999e-05 2024-09-15T01:32:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231591+0000", 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015873015000000001 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231666+0000", 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5525999999999995e-05 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233961+0000", 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022948069999999998 2024-09-15T01:32:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:93 3.7s0 3:e6e8875f:::smithi17529310-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.459364665999999, 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079343245000000007, 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 93 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1299999999999999e-07 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229390+0000", 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.703e-06 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229403+0000", 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3056e-05 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229410+0000", 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231135+0000", 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001725737 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231216+0000", 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0586000000000004e-05 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233251+0000", 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020346819999999999 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233339+0000", 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7907999999999999e-05 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308729+0000", 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.075390190999999995 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:306 3.7s0 3:e6e8875f:::smithi17529310-31:head [append 0~0] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.269780+0000", 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.418970597, 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013438767000000001, 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 306 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269780+0000", 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269780+0000", 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269782+0000", 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3489999999999999e-06 2024-09-15T01:32:56.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269788+0000", 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1589999999999998e-06 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269806+0000", 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7642000000000002e-05 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269813+0000", 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6159999999999996e-06 2024-09-15T01:32:56.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269878+0000", 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4121999999999994e-05 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270069+0000", 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000191769 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270780+0000", 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071044299999999995 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271660+0000", 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00088018600000000003 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271671+0000", 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1258e-05 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271715+0000", 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4091000000000002e-05 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272505+0000", 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00078956700000000002 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.283171+0000", 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010666261 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.283218+0000", 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7304e-05 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:307 3.7s0 3:e6e8875f:::smithi17529310-31:head [rollback 8] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.270030+0000", 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.41872038, 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013327521, 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 307 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270030+0000", 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270030+0000", 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270031+0000", 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9000000000000005e-07 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270035+0000", 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1300000000000003e-06 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270046+0000", 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0661e-05 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270054+0000", 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0039999999999999e-06 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271724+0000", 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001670232 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271818+0000", 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4290999999999996e-05 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272296+0000", 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047796699999999999 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272405+0000", 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109005 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272420+0000", 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5184000000000001e-05 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272460+0000", 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9787999999999999e-05 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272511+0000", 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0872999999999999e-05 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.283309+0000", 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010797676000000001 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.283357+0000", 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8720000000000001e-05 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:308 3.7s0 3:e6e8875f:::smithi17529310-31:head [append 0~0] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.270062+0000", 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.418687774, 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013400334, 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 308 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270062+0000", 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270062+0000", 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270063+0000", 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6100000000000001e-07 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270065+0000", 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5909999999999998e-06 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270073+0000", 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9680000000000006e-06 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.270096+0000", 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2739e-05 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272644+0000", 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025474 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272741+0000", 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7428000000000003e-05 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.273173+0000", 2024-09-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043247400000000002 2024-09-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.273245+0000", 2024-09-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1952000000000005e-05 2024-09-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.274248+0000", 2024-09-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010024890000000001 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.274289+0000", 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0633000000000002e-05 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.274306+0000", 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7827000000000001e-05 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.283424+0000", 2024-09-15T01:32:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091179590000000001 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.283463+0000", 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8413e-05 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.7s0 47/38 ECSubWriteReply(tid=76, last_complete=47'25, committed=1, applied=1))", 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.324527657, 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.42564722900000002, 2024-09-15T01:32:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8599999999999999e-07 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364224+0000", 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5099999999999999e-06 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789796+0000", 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.425571957 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789805+0000", 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789857+0000", 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1598999999999998e-05 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789870+0000", 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2850000000000001e-05 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpReadReply(3.1s0 53/52 ECSubReadReply(tid=94, attrs_read=0))", 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:55.205063+0000", 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.483686727, 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99844881399999996, 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:55.205063+0000", 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:55.205063+0000", 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:55.205064+0000", 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:55.205072+0000", 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5299999999999996e-06 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.202784+0000", 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99771169100000001 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.202787+0000", 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.841e-06 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.202869+0000", 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1705999999999993e-05 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.203512+0000", 2024-09-15T01:32:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064375100000000002 2024-09-15T01:32:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:500 3.7s0 3:e89d09e8:::smithi17529310-19:head [setxattr _header (57) in=64b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:32:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.209208+0000", 2024-09-15T01:32:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.47954198799999997, 2024-09-15T01:32:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.280345175, 2024-09-15T01:32:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 500 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209208+0000", 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209208+0000", 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209209+0000", 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4199999999999995e-07 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209213+0000", 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8410000000000002e-06 2024-09-15T01:32:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209217+0000", 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.391e-06 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209220+0000", 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6570000000000001e-06 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.210612+0000", 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013926850000000001 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.210830+0000", 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000217655 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211263+0000", 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043276600000000001 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211409+0000", 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014583400000000001 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211418+0000", 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7459999999999995e-06 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211458+0000", 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999999e-05 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211653+0000", 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000195801 2024-09-15T01:32:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.489314+0000", 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.27766080300000001 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.489553+0000", 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023910399999999999 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:501 3.7s0 3:e89d09e8:::smithi17529310-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.47952763100000001, 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.28201526999999998, 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 501 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:32:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209225+0000", 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9819999999999998e-06 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209229+0000", 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3959999999999999e-06 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209233+0000", 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5609999999999999e-06 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211492+0000", 2024-09-15T01:32:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002259414 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211637+0000", 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014431099999999999 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.489907+0000", 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.27827002099999998 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.490070+0000", 2024-09-15T01:32:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016353799999999999 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.491238+0000", 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011676309999999999 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.7s0 54/52 ECSubWriteReply(tid=119, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.211852+0000", 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.47689827000000001, 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19905882499999999, 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211852+0000", 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211852+0000", 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211853+0000", 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8500000000000001e-07 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211858+0000", 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9049999999999996e-06 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.410806+0000", 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.198948825 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.410818+0000", 2024-09-15T01:32:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1161e-05 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.410890+0000", 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2223000000000003e-05 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.410911+0000", 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1026000000000002e-05 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:56.802 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:56.803 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:32:56.803 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:32:56.802+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:32:56.903 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:56.902+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:56.904 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:32:56.904 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:56.912 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:32:56.922 INFO:tasks.rados.rados.0.smithi175.stdout:174: finishing write tid 5 to smithi17529310-12 2024-09-15T01:32:56.923 INFO:tasks.rados.rados.0.smithi175.stdout:174: finishing write tid 6 to smithi17529310-12 2024-09-15T01:32:56.923 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 12 v 49 (ObjNum 98 snap 10 seq_num 98) dirty exists 2024-09-15T01:32:56.923 INFO:tasks.rados.rados.0.smithi175.stdout:174: left oid 12 (ObjNum 98 snap 10 seq_num 98) 2024-09-15T01:32:57.004 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:57.003+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:57.012 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:57.013 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:57.013 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:32:57.013 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:32:57.013 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:57.021 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:32:57.069 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.15 to 1 (10000) 2024-09-15T01:32:57.074 INFO:tasks.rados.rados.0.smithi175.stdout:165: done (8 left) 2024-09-15T01:32:57.074 INFO:tasks.rados.rados.0.smithi175.stdout:174: done (7 left) 2024-09-15T01:32:57.074 INFO:tasks.rados.rados.0.smithi175.stdout:179: done (6 left) 2024-09-15T01:32:57.074 INFO:tasks.rados.rados.0.smithi175.stdout:180: done (5 left) 2024-09-15T01:32:57.074 INFO:tasks.rados.rados.0.smithi175.stdout:181: done (4 left) 2024-09-15T01:32:57.074 INFO:tasks.rados.rados.0.smithi175.stdout:183: done (3 left) 2024-09-15T01:32:57.074 INFO:tasks.rados.rados.0.smithi175.stdout:184: done (2 left) 2024-09-15T01:32:57.074 INFO:tasks.rados.rados.0.smithi175.stdout:185: done (1 left) 2024-09-15T01:32:57.074 INFO:tasks.rados.rados.0.smithi175.stdout:186: done (0 left) 2024-09-15T01:32:57.074 INFO:tasks.rados.rados.0.smithi175.stdout:187: copy_from oid 42 from oid 19 current snap is 11 2024-09-15T01:32:57.074 INFO:tasks.rados.rados.0.smithi175.stdout:188: copy_from oid 4 from oid 8 current snap is 11 2024-09-15T01:32:57.075 INFO:tasks.rados.rados.0.smithi175.stdout:189: copy_from oid 7 from oid 25 current snap is 11 2024-09-15T01:32:57.075 INFO:tasks.rados.rados.0.smithi175.stdout:190: copy_from oid 45 from oid 29 current snap is 11 2024-09-15T01:32:57.075 INFO:tasks.rados.rados.0.smithi175.stdout:191: rollback oid 48 current snap is 11 2024-09-15T01:32:57.075 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 48 to 10 2024-09-15T01:32:57.075 INFO:tasks.rados.rados.0.smithi175.stdout:192: copy_from oid 49 from oid 28 current snap is 11 2024-09-15T01:32:57.075 INFO:tasks.rados.rados.0.smithi175.stdout:193: rollback oid 18 current snap is 11 2024-09-15T01:32:57.075 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 18 to 10 2024-09-15T01:32:57.075 INFO:tasks.rados.rados.0.smithi175.stdout:194: snap_remove snap 9 2024-09-15T01:32:57.091 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T01:32:57.091 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:32:57.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T01:32:57.105 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:57.104+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.5s1 33/31 ECSubWrite(tid=1, reqid=client.4477.0:11, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.955389516, 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030602694, 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8900000000000001e-07 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178311+0000", 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5959999999999993e-06 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208281+0000", 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029970173999999999 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208291+0000", 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3850000000000008e-06 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208407+0000", 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116168 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208422+0000", 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4931e-05 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208519+0000", 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7213999999999995e-05 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208541+0000", 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2249999999999999e-05 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208644+0000", 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102537 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208664+0000", 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.071e-05 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208904+0000", 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023944 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.5s1 34/31 ECSubRead(tid=6, to_read={3:af8bec4d:::smithi17529310-24:head=0,4096,0}, subchunks={3:af8bec4d:::smithi17529310-24:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.210749+0000", 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.922941640000001, 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033380170000000001, 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210749+0000", 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210749+0000", 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210750+0000", 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6000000000000004e-07 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210754+0000", 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7049999999999998e-06 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.243939+0000", 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033185092999999999 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.243951+0000", 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1864e-05 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.244010+0000", 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8536000000000003e-05 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.244129+0000", 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119412 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/31 ECSubWrite(tid=22, reqid=client.4477.0:113, at_version=34'10, trim_to=0'0, roll_forward_to=34'7))", 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:32:57.145 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.878471767000001, 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035418835000000003, 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255233+0000", 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3815000000000001e-05 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.289999+0000", 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.034765375000000001 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290004+0000", 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290062+0000", 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7763999999999998e-05 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290072+0000", 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7829999999999993e-06 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290165+0000", 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3039000000000001e-05 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290192+0000", 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7449000000000001e-05 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290303+0000", 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110253 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290375+0000", 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2855999999999995e-05 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290638+0000", 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026231300000000002 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:128 3.4s0 3:299c4018:::smithi17529310-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.275429+0000", 2024-09-15T01:32:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.858261665000001, 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.10370515499999999, 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 128 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275429+0000", 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275429+0000", 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275430+0000", 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1099999999999995e-07 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275434+0000", 2024-09-15T01:32:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.42e-06 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275443+0000", 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1330000000000005e-06 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275448+0000", 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3769999999999998e-06 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.276441+0000", 2024-09-15T01:32:57.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099363500000000009 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.276580+0000", 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013892099999999999 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277288+0000", 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070808500000000001 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277385+0000", 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6824999999999995e-05 2024-09-15T01:32:57.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277394+0000", 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0839999999999998e-06 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277434+0000", 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9465999999999998e-05 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277551+0000", 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117162 2024-09-15T01:32:57.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379048+0000", 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101497254 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379134+0000", 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6081999999999993e-05 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.156 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:129 3.4s0 3:299c4018:::smithi17529310-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.858221824000001, 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.105118901, 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 129 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:32:57.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8999999999999996e-07 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275472+0000", 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6419999999999999e-06 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275480+0000", 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0790000000000001e-06 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275485+0000", 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4689999999999999e-06 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277457+0000", 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019728160000000001 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277534+0000", 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6073000000000002e-05 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379464+0000", 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.10193010499999999 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379556+0000", 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2084999999999997e-05 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.380588+0000", 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010320419999999999 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.cs1 34/31 ECSubRead(tid=37, to_read={3:34a45b5f:::smithi17529310-48:head=0,4096,0}, subchunks={3:34a45b5f:::smithi17529310-48:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.825856009999999, 2024-09-15T01:32:57.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14311833299999999, 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6499999999999999e-07 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307844+0000", 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4810000000000006e-06 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450796+0000", 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14295194899999999 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450806+0000", 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7890000000000006e-06 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450858+0000", 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2577999999999997e-05 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450953+0000", 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4870999999999993e-05 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:217 3.4s0 3:299c4018:::smithi17529310-43:head [copy-from ver 4] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.208686+0000", 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.925004591, 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013624864, 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 217 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.165 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208686+0000", 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208686+0000", 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208688+0000", 2024-09-15T01:32:57.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3989999999999998e-06 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208709+0000", 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0849999999999999e-05 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208729+0000", 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9681e-05 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208738+0000", 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5350000000000004e-06 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208841+0000", 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103353 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208941+0000", 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9986000000000001e-05 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.219554+0000", 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01061337 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.219922+0000", 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036728800000000001 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.219935+0000", 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3689000000000001e-05 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220006+0000", 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1073000000000002e-05 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220034+0000", 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7736e-05 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.221399+0000", 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013649000000000001 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.222311+0000", 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000912004 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:218 3.4s0 3:299c4018:::smithi17529310-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.924938055, 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011546681, 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:57.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 218 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6799999999999999e-07 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208757+0000", 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.941e-06 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208767+0000", 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8509999999999994e-06 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208786+0000", 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8320000000000001e-05 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209130+0000", 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034496100000000002 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209184+0000", 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3703999999999999e-05 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220077+0000", 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010893277999999999 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220195+0000", 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117342 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220299+0000", 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104516 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.9s5 44/31 ECSubWrite(tid=60, reqid=osd.14.0:60, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.238474+0000", 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.895216752, 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010306067, 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.238474+0000", 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.238474+0000", 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.238474+0000", 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.238499+0000", 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4298999999999999e-05 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.238558+0000", 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9790999999999997e-05 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.238570+0000", 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2001000000000001e-05 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.239089+0000", 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00051868599999999997 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.239101+0000", 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1545999999999999e-05 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240653+0000", 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015523399999999999 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240922+0000", 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026856699999999999 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241196+0000", 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027480400000000002 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241239+0000", 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2919e-05 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248780+0000", 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075407540000000002 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs1 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.241172+0000", 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.892518339, 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018176115, 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241172+0000", 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241172+0000", 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241173+0000", 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.05e-07 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241195+0000", 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2033000000000001e-05 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241238+0000", 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3507999999999997e-05 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241245+0000", 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2509999999999997e-06 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245590+0000", 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043445309999999996 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245599+0000", 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8519999999999993e-06 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246349+0000", 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00075022999999999995 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246362+0000", 2024-09-15T01:32:57.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.273e-05 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246399+0000", 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7299000000000003e-05 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246806+0000", 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040714800000000003 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.259348+0000", 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012542328 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs1 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.890425097, 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017149162999999999, 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243294+0000", 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7868000000000002e-05 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243334+0000", 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9369000000000002e-05 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243341+0000", 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7009999999999996e-06 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246522+0000", 2024-09-15T01:32:57.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031802359999999999 2024-09-15T01:32:57.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246530+0000", 2024-09-15T01:32:57.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5019999999999992e-06 2024-09-15T01:32:57.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246730+0000", 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019988499999999999 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246745+0000", 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4632e-05 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246784+0000", 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9072000000000001e-05 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246850+0000", 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699000000000006e-05 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.260415+0000", 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013564296 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 44/31 ECSubWrite(tid=82, reqid=client.4477.0:281, at_version=44'36, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.244090+0000", 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.889600401999999, 2024-09-15T01:32:57.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0099664520000000006, 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244090+0000", 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244090+0000", 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244091+0000", 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244125+0000", 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3414000000000002e-05 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244173+0000", 2024-09-15T01:32:57.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8111000000000001e-05 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244180+0000", 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2010000000000002e-06 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244245+0000", 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4628999999999994e-05 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244254+0000", 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3340000000000008e-06 2024-09-15T01:32:57.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245520+0000", 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012662210000000001 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245531+0000", 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.03e-05 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245586+0000", 2024-09-15T01:32:57.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5575999999999998e-05 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245878+0000", 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029204099999999999 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.254057+0000", 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081785069999999998 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as3 44/38 ECSubWrite(tid=25, reqid=osd.3.0:25, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.244320+0000", 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.889371012, 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010577385999999999, 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244320+0000", 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244320+0000", 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244320+0000", 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244344+0000", 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4134e-05 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244427+0000", 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2767000000000002e-05 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244434+0000", 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6610000000000003e-06 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244924+0000", 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049050600000000004 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244934+0000", 2024-09-15T01:32:57.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1889999999999996e-06 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246070+0000", 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001135973 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246098+0000", 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.809e-05 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246133+0000", 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5670000000000002e-05 2024-09-15T01:32:57.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246146+0000", 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3141e-05 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.254897+0000", 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087505680000000002 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 44/31 ECSubWrite(tid=83, reqid=osd.11.0:83, at_version=44'38, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.245401+0000", 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.888289197000001, 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0098909730000000008, 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245401+0000", 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245401+0000", 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245402+0000", 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2399999999999997e-07 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245424+0000", 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1650000000000001e-05 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245456+0000", 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2016e-05 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245462+0000", 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.964e-06 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245579+0000", 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117553 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245589+0000", 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1370000000000008e-06 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245842+0000", 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025340999999999997 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245853+0000", 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0712e-05 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245924+0000", 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1120999999999999e-05 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245935+0000", 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1208e-05 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.255292+0000", 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0093574780000000007 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds5 44/31 ECSubWrite(tid=36, reqid=osd.7.0:36, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.887700096, 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013253133, 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6199999999999999e-07 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246013+0000", 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1801e-05 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246123+0000", 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109879 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246132+0000", 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8670000000000006e-06 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246240+0000", 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107211 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246248+0000", 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8480000000000007e-06 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246452+0000", 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020319399999999999 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246461+0000", 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-15T01:32:57.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246492+0000", 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1124999999999998e-05 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246831+0000", 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033878400000000001 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.259244+0000", 2024-09-15T01:32:57.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012412444 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.246909+0000", 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.886782068, 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013946906, 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246909+0000", 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246909+0000", 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246910+0000", 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.16e-07 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246933+0000", 2024-09-15T01:32:57.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3884e-05 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246981+0000", 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7194000000000002e-05 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246988+0000", 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0260000000000002e-06 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247055+0000", 2024-09-15T01:32:57.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7216000000000002e-05 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247066+0000", 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1562999999999999e-05 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248672+0000", 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001605455 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248697+0000", 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5588e-05 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248730+0000", 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2351000000000002e-05 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248812+0000", 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2254000000000002e-05 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.260856+0000", 2024-09-15T01:32:57.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012043458999999999 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds5 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.886564181000001, 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014066664, 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4000000000000001e-07 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247148+0000", 2024-09-15T01:32:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1339e-05 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247188+0000", 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9301000000000002e-05 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247196+0000", 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2789999999999998e-06 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247453+0000", 2024-09-15T01:32:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025692899999999998 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247463+0000", 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0317e-05 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248725+0000", 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012613730000000001 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248732+0000", 2024-09-15T01:32:57.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9129999999999997e-06 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248776+0000", 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4280000000000003e-05 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248811+0000", 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5553000000000003e-05 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.261193+0000", 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01238174 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as3 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.885830930000001, 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012019658000000001, 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247861+0000", 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6700000000000002e-07 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247872+0000", 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1307e-05 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247909+0000", 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7290999999999999e-05 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247917+0000", 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3590000000000002e-06 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248032+0000", 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115566 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248048+0000", 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6069000000000001e-05 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248184+0000", 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013525399999999999 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248205+0000", 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1021000000000001e-05 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248244+0000", 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9847e-05 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248260+0000", 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5639000000000001e-05 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.259879+0000", 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011619338 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds5 44/31 ECSubWrite(tid=38, reqid=osd.7.0:37, at_version=0'0, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.251858+0000", 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.881832623999999, 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010171173, 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251858+0000", 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251858+0000", 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251943+0000", 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4887000000000002e-05 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252362+0000", 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0004193 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252452+0000", 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0026999999999997e-05 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252460+0000", 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0080000000000002e-06 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252528+0000", 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7414000000000002e-05 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252538+0000", 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.023e-05 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252674+0000", 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000136017 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252704+0000", 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0368000000000001e-05 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253070+0000", 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000365204 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253092+0000", 2024-09-15T01:32:57.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2693000000000001e-05 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.262029+0000", 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089370249999999995 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:292 3.4s0 3:299c4018:::smithi17529310-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.267708+0000", 2024-09-15T01:32:57.232 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.86598279, 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020077541000000001, 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 292 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267708+0000", 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267708+0000", 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267709+0000", 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7800000000000001e-07 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267712+0000", 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-15T01:32:57.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267723+0000", 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1388e-05 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267729+0000", 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5890000000000002e-06 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269684+0000", 2024-09-15T01:32:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019553389999999999 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269888+0000", 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020362599999999999 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.273250+0000", 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033616280000000002 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.273338+0000", 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7990000000000003e-05 2024-09-15T01:32:57.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.287785+0000", 2024-09-15T01:32:57.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01444779 2024-09-15T01:32:57.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.237 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.237 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.237 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:57.238 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-15T01:32:57.239 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:57.237+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:57.339 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:32:57.337+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:32:57.382 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:57.382 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:57.382 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:32:57.382 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:57.384 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:57.385 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:32:57.392 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-15T01:32:57.397 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-09-15T01:32:57.076334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":55460403,"num_objects":54,"num_object_clones":9,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":1,"num_read":300,"num_read_kb":28806,"num_write":506,"num_write_kb":70274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":9879552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":607,"ondisk_log_size":607,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":15,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1405194240,"kb_used":12117760,"kb_used_data":1393092464,"kb_used_omap":12198,"kb_used_meta":0,"kb_avail":1393076480,"statfs":{"total":1438918901760,"available":1426510315520,"internally_reserved":16367616,"allocated":1426526683136,"data_stored":1426526683136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12491579,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":4,"commit_latency_ns":0,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5341184,"num_objects":-3,"num_object_clones":-4,"num_object_copies":-21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3,"num_whiteouts":1,"num_read":19,"num_read_kb":8633,"num_write":63,"num_write_kb":10448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5832704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000869"},"pg_stats":[{"pgid":"3.f","version":"52'70","reported_seq":137,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.159541+0000","last_change":"2024-09-15T01:32:54.083572+0000","last_active":"2024-09-15T01:32:54.159541+0000","last_peered":"2024-09-15T01:32:54.159541+0000","last_clean":"2024-09-15T01:32:54.159541+0000","last_became_active":"2024-09-15T01:32:34.487273+0000","last_became_peered":"2024-09-15T01:32:34.487273+0000","last_unstale":"2024-09-15T01:32:54.159541+0000","last_undegraded":"2024-09-15T01:32:54.159541+0000","last_fullsized":"2024-09-15T01:32:54.159541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:42.230488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072134800000000002,"stat_sum":{"num_bytes":4456448,"num_objects":7,"num_object_clones":2,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":23,"num_read_kb":149,"num_write":71,"num_write_kb":7776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"2"},{"start":"f","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.079870+0000","last_change":"2024-09-15T01:32:55.080194+0000","last_active":"2024-09-15T01:32:55.079870+0000","last_peered":"2024-09-15T01:32:55.079870+0000","last_clean":"2024-09-15T01:32:55.079870+0000","last_became_active":"2024-09-15T01:32:54.091710+0000","last_became_peered":"2024-09-15T01:32:54.091710+0000","last_unstale":"2024-09-15T01:32:55.079870+0000","last_undegraded":"2024-09-15T01:32:55.079870+0000","last_fullsized":"2024-09-15T01:32:55.079870+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:55.820742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067712999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,14,1,11,8,6,12],"acting":[2,14,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.d","version":"51'42","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.081416+0000","last_change":"2024-09-15T01:32:55.081742+0000","last_active":"2024-09-15T01:32:55.081416+0000","last_peered":"2024-09-15T01:32:55.081416+0000","last_clean":"2024-09-15T01:32:55.081416+0000","last_became_active":"2024-09-15T01:32:34.487040+0000","last_became_peered":"2024-09-15T01:32:34.487040+0000","last_unstale":"2024-09-15T01:32:55.081416+0000","last_undegraded":"2024-09-15T01:32:55.081416+0000","last_fullsized":"2024-09-15T01:32:55.081416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.966217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047890500000000001,"stat_sum":{"num_bytes":2473984,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":871,"num_write":19,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.c","version":"51'50","reported_seq":120,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.080667+0000","last_change":"2024-09-15T01:32:54.082408+0000","last_active":"2024-09-15T01:32:54.080667+0000","last_peered":"2024-09-15T01:32:54.080667+0000","last_clean":"2024-09-15T01:32:54.080667+0000","last_became_active":"2024-09-15T01:32:34.487072+0000","last_became_peered":"2024-09-15T01:32:34.487072+0000","last_unstale":"2024-09-15T01:32:54.080667+0000","last_undegraded":"2024-09-15T01:32:54.080667+0000","last_fullsized":"2024-09-15T01:32:54.080667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.186197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022585959999999999,"stat_sum":{"num_bytes":7864320,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":31,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"2"},{"start":"f","length":"1"}]},{"pgid":"3.b","version":"53'23","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.086294+0000","last_change":"2024-09-15T01:32:55.087737+0000","last_active":"2024-09-15T01:32:55.086294+0000","last_peered":"2024-09-15T01:32:55.086294+0000","last_clean":"2024-09-15T01:32:55.086294+0000","last_became_active":"2024-09-15T01:32:41.206211+0000","last_became_peered":"2024-09-15T01:32:41.206211+0000","last_unstale":"2024-09-15T01:32:55.086294+0000","last_undegraded":"2024-09-15T01:32:55.086294+0000","last_fullsized":"2024-09-15T01:32:55.086294+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.489551+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0087153419999999992,"stat_sum":{"num_bytes":2457600,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":10,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,13],"acting":[4,0,1,12,6,7,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.a","version":"44'19","reported_seq":123,"reported_epoch":53,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-15T01:32:55.086121+0000","last_change":"2024-09-15T01:32:55.082208+0000","last_active":"2024-09-15T01:32:53.073499+0000","last_peered":"2024-09-15T01:32:52.073534+0000","last_clean":"2024-09-15T01:32:52.073534+0000","last_became_active":"2024-09-15T01:32:41.201415+0000","last_became_peered":"2024-09-15T01:32:41.201415+0000","last_unstale":"2024-09-15T01:32:55.086121+0000","last_undegraded":"2024-09-15T01:32:55.080312+0000","last_fullsized":"2024-09-15T01:32:55.079754+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.689368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062814200000000004,"stat_sum":{"num_bytes":1654784,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":9,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8,14,6,11],"acting":[3,7,1,8,14,6,2147483647],"avail_no_missing":["3(0)","1(2)","6(5)","7(1)","8(3)","10(6)","14(4)"],"object_location_counts":[{"shards":"1(2),3(0),6(5),7(1),8(3),10(6),14(4)","objects":1}],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"1"}]},{"pgid":"3.9","version":"51'34","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.081224+0000","last_change":"2024-09-15T01:32:55.081698+0000","last_active":"2024-09-15T01:32:55.081224+0000","last_peered":"2024-09-15T01:32:55.081224+0000","last_clean":"2024-09-15T01:32:55.081224+0000","last_became_active":"2024-09-15T01:32:34.487117+0000","last_became_peered":"2024-09-15T01:32:34.487117+0000","last_unstale":"2024-09-15T01:32:55.081224+0000","last_undegraded":"2024-09-15T01:32:55.081224+0000","last_fullsized":"2024-09-15T01:32:55.081224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.760814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001249448,"stat_sum":{"num_bytes":4849664,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":2601,"num_write":19,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.8","version":"51'43","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.077556+0000","last_change":"2024-09-15T01:32:55.078177+0000","last_active":"2024-09-15T01:32:55.077556+0000","last_peered":"2024-09-15T01:32:55.077556+0000","last_clean":"2024-09-15T01:32:55.077556+0000","last_became_active":"2024-09-15T01:32:34.190371+0000","last_became_peered":"2024-09-15T01:32:34.190371+0000","last_unstale":"2024-09-15T01:32:55.077556+0000","last_undegraded":"2024-09-15T01:32:55.077556+0000","last_fullsized":"2024-09-15T01:32:55.077556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:51.142885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077278100000000005,"stat_sum":{"num_bytes":6258688,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":44,"num_write_kb":6112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"1.0","version":"24'112","reported_seq":173,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.077251+0000","last_change":"2024-09-15T01:32:16.749959+0000","last_active":"2024-09-15T01:32:54.077251+0000","last_peered":"2024-09-15T01:32:54.077251+0000","last_clean":"2024-09-15T01:32:54.077251+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:32:54.077251+0000","last_undegraded":"2024-09-15T01:32:54.077251+0000","last_fullsized":"2024-09-15T01:32:54.077251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_clean_scrub_stamp":"2024-09-15T01:32:15.451214+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.075494+0000","last_change":"2024-09-15T01:32:25.147954+0000","last_active":"2024-09-15T01:32:54.075494+0000","last_peered":"2024-09-15T01:32:54.075494+0000","last_clean":"2024-09-15T01:32:54.075494+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:32:54.075494+0000","last_undegraded":"2024-09-15T01:32:54.075494+0000","last_fullsized":"2024-09-15T01:32:54.075494+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.228776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"53'24","reported_seq":116,"reported_epoch":53,"state":"active+undersized+remapped","last_fresh":"2024-09-15T01:32:55.231718+0000","last_change":"2024-09-15T01:32:55.232233+0000","last_active":"2024-09-15T01:32:55.231718+0000","last_peered":"2024-09-15T01:32:55.231718+0000","last_clean":"2024-09-15T01:32:52.098439+0000","last_became_active":"2024-09-15T01:32:55.189455+0000","last_became_peered":"2024-09-15T01:32:55.189455+0000","last_unstale":"2024-09-15T01:32:55.231718+0000","last_undegraded":"2024-09-15T01:32:55.078442+0000","last_fullsized":"2024-09-15T01:32:55.077832+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:48.962933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0085917170000000005,"stat_sum":{"num_bytes":4341760,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":14,"num_read_kb":3846,"num_write":17,"num_write_kb":5776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":5652480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,2147483647],"avail_no_missing":["13(0)","1(5)","2(1)","3(2)","6(3)","11(6)","14(6)","15(4)"],"object_location_counts":[{"shards":"1(5),2(1),3(2),6(3),13(0),15(4)","objects":5}],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.076773+0000","last_change":"2024-09-15T01:32:25.143709+0000","last_active":"2024-09-15T01:32:54.076773+0000","last_peered":"2024-09-15T01:32:54.076773+0000","last_clean":"2024-09-15T01:32:54.076773+0000","last_became_active":"2024-09-15T01:32:23.134382+0000","last_became_peered":"2024-09-15T01:32:23.134382+0000","last_unstale":"2024-09-15T01:32:54.076773+0000","last_undegraded":"2024-09-15T01:32:54.076773+0000","last_fullsized":"2024-09-15T01:32:54.076773+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.326224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"53'8","reported_seq":126,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T01:32:55.207384+0000","last_change":"2024-09-15T01:32:55.205526+0000","last_active":"2024-09-15T01:32:55.207384+0000","last_peered":"2024-09-15T01:32:55.207384+0000","last_clean":"2024-09-15T01:32:52.072110+0000","last_became_active":"2024-09-15T01:32:55.189409+0000","last_became_peered":"2024-09-15T01:32:55.189409+0000","last_unstale":"2024-09-15T01:32:55.207384+0000","last_undegraded":"2024-09-15T01:32:55.079520+0000","last_fullsized":"2024-09-15T01:32:55.078828+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:47.882206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063910499999999995,"stat_sum":{"num_bytes":1867776,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1058,"num_write":8,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":442368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,15,5,10,2,13],"acting":[4,2147483647,15,5,10,2,13],"avail_no_missing":["4(0)","2(5)","5(3)","7(1)","10(4)","13(6)","15(2)"],"object_location_counts":[{"shards":"2(5),4(0),5(3),10(4),13(6),15(2)","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.077416+0000","last_change":"2024-09-15T01:32:25.583876+0000","last_active":"2024-09-15T01:32:54.077416+0000","last_peered":"2024-09-15T01:32:54.077416+0000","last_clean":"2024-09-15T01:32:54.077416+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:32:54.077416+0000","last_undegraded":"2024-09-15T01:32:54.077416+0000","last_fullsized":"2024-09-15T01:32:54.077416+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:49.092006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"53'34","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.085333+0000","last_change":"2024-09-15T01:32:55.085991+0000","last_active":"2024-09-15T01:32:55.085333+0000","last_peered":"2024-09-15T01:32:55.085333+0000","last_clean":"2024-09-15T01:32:55.085333+0000","last_became_active":"2024-09-15T01:32:34.190384+0000","last_became_peered":"2024-09-15T01:32:34.190384+0000","last_unstale":"2024-09-15T01:32:55.085333+0000","last_undegraded":"2024-09-15T01:32:55.085333+0000","last_fullsized":"2024-09-15T01:32:55.085333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.588055+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.004685486,"stat_sum":{"num_bytes":1703936,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":341,"num_write":14,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"2.2","version":"28'2","reported_seq":42,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.077457+0000","last_change":"2024-09-15T01:32:25.144113+0000","last_active":"2024-09-15T01:32:54.077457+0000","last_peered":"2024-09-15T01:32:54.077457+0000","last_clean":"2024-09-15T01:32:54.077457+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:32:54.077457+0000","last_undegraded":"2024-09-15T01:32:54.077457+0000","last_fullsized":"2024-09-15T01:32:54.077457+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"53'34","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.082017+0000","last_change":"2024-09-15T01:32:55.083145+0000","last_active":"2024-09-15T01:32:55.082017+0000","last_peered":"2024-09-15T01:32:55.082017+0000","last_clean":"2024-09-15T01:32:55.082017+0000","last_became_active":"2024-09-15T01:32:34.170218+0000","last_became_peered":"2024-09-15T01:32:34.170218+0000","last_unstale":"2024-09-15T01:32:55.082017+0000","last_undegraded":"2024-09-15T01:32:55.082017+0000","last_fullsized":"2024-09-15T01:32:55.082017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.093067+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0045204169999999997,"stat_sum":{"num_bytes":2867200,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":1544,"num_write":37,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.5","version":"52'24","reported_seq":97,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.180134+0000","last_change":"2024-09-15T01:32:54.083046+0000","last_active":"2024-09-15T01:32:54.180134+0000","last_peered":"2024-09-15T01:32:54.180134+0000","last_clean":"2024-09-15T01:32:54.180134+0000","last_became_active":"2024-09-15T01:32:34.189813+0000","last_became_peered":"2024-09-15T01:32:34.189813+0000","last_unstale":"2024-09-15T01:32:54.180134+0000","last_undegraded":"2024-09-15T01:32:54.180134+0000","last_fullsized":"2024-09-15T01:32:54.180134+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:47.814751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050507099999999995,"stat_sum":{"num_bytes":3637248,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":6006,"num_write":15,"num_write_kb":3936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"2"},{"start":"f","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.077209+0000","last_change":"2024-09-15T01:32:25.568260+0000","last_active":"2024-09-15T01:32:54.077209+0000","last_peered":"2024-09-15T01:32:54.077209+0000","last_clean":"2024-09-15T01:32:54.077209+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:32:54.077209+0000","last_undegraded":"2024-09-15T01:32:54.077209+0000","last_fullsized":"2024-09-15T01:32:54.077209+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:32.165806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"45'31","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.083986+0000","last_change":"2024-09-15T01:32:55.084335+0000","last_active":"2024-09-15T01:32:55.083986+0000","last_peered":"2024-09-15T01:32:55.083986+0000","last_clean":"2024-09-15T01:32:55.083986+0000","last_became_active":"2024-09-15T01:32:34.189644+0000","last_became_peered":"2024-09-15T01:32:34.189644+0000","last_unstale":"2024-09-15T01:32:55.083986+0000","last_undegraded":"2024-09-15T01:32:55.083986+0000","last_fullsized":"2024-09-15T01:32:55.083986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:02.903663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049015899999999999,"stat_sum":{"num_bytes":2359296,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":4567,"num_write":17,"num_write_kb":4528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.081132+0000","last_change":"2024-09-15T01:32:25.407396+0000","last_active":"2024-09-15T01:32:54.081132+0000","last_peered":"2024-09-15T01:32:54.081132+0000","last_clean":"2024-09-15T01:32:54.081132+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:32:54.081132+0000","last_undegraded":"2024-09-15T01:32:54.081132+0000","last_fullsized":"2024-09-15T01:32:54.081132+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:39.303667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"50'31","reported_seq":148,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T01:32:55.189951+0000","last_change":"2024-09-15T01:32:55.190604+0000","last_active":"2024-09-15T01:32:55.189951+0000","last_peered":"2024-09-15T01:32:55.189951+0000","last_clean":"2024-09-15T01:32:52.088276+0000","last_became_active":"2024-09-15T01:32:55.189564+0000","last_became_peered":"2024-09-15T01:32:55.189564+0000","last_unstale":"2024-09-15T01:32:55.189951+0000","last_undegraded":"2024-09-15T01:32:55.079592+0000","last_fullsized":"2024-09-15T01:32:55.078832+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:45.333547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015950892000000001,"stat_sum":{"num_bytes":5980160,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":917,"num_write":18,"num_write_kb":5840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,0,8,5,11],"acting":[4,6,1,0,8,5,2147483647],"avail_no_missing":["4(0)","0(3)","1(2)","5(5)","6(1)","8(4)","14(6)"],"object_location_counts":[{"shards":"0(3),1(2),4(0),5(5),6(1),8(4),14(6)","objects":7}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.080321+0000","last_change":"2024-09-15T01:32:25.568523+0000","last_active":"2024-09-15T01:32:54.080321+0000","last_peered":"2024-09-15T01:32:54.080321+0000","last_clean":"2024-09-15T01:32:54.080321+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:32:54.080321+0000","last_undegraded":"2024-09-15T01:32:54.080321+0000","last_fullsized":"2024-09-15T01:32:54.080321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"51'25","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.084087+0000","last_change":"2024-09-15T01:32:55.084545+0000","last_active":"2024-09-15T01:32:55.084087+0000","last_peered":"2024-09-15T01:32:55.084087+0000","last_clean":"2024-09-15T01:32:55.084087+0000","last_became_active":"2024-09-15T01:32:41.203815+0000","last_became_peered":"2024-09-15T01:32:41.203815+0000","last_unstale":"2024-09-15T01:32:55.084087+0000","last_undegraded":"2024-09-15T01:32:55.084087+0000","last_fullsized":"2024-09-15T01:32:55.084087+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:48.344390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066820499999999995,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":6811,"num_write":30,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,10],"acting":[0,13,1,15,12,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"2.7","version":"0'0","reported_seq":38,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T01:32:53.075475+0000","last_change":"2024-09-15T01:32:25.582726+0000","last_active":"2024-09-15T01:32:53.075475+0000","last_peered":"2024-09-15T01:32:53.075475+0000","last_clean":"2024-09-15T01:32:53.075475+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:32:53.075475+0000","last_undegraded":"2024-09-15T01:32:53.075475+0000","last_fullsized":"2024-09-15T01:32:53.075475+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.907962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":54870016,"num_objects":50,"num_object_clones":9,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":1,"num_read":214,"num_read_kb":28733,"num_write":359,"num_write_kb":67600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":9879552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":492,"ondisk_log_size":492,"up":112,"acting":112,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313228,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":811528,"kb_used_data":92868916,"kb_used_omap":1037,"kb_used_meta":0,"kb_avail":92868088,"statfs":{"total":95927926784,"available":95096922112,"internally_reserved":847872,"allocated":95097769984,"data_stored":95097769984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1062342,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":9,"up_from":0,"seq":90194313226,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":21,"seq":90194313227,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":803352,"kb_used_data":92877000,"kb_used_omap":476,"kb_used_meta":0,"kb_avail":92876264,"statfs":{"total":95927926784,"available":95105294336,"internally_reserved":753664,"allocated":95106048000,"data_stored":95106048000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":487440,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":0,"up_from":21,"seq":90194313228,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":814304,"kb_used_data":92866808,"kb_used_omap":1281,"kb_used_meta":0,"kb_avail":92865312,"statfs":{"total":95927926784,"available":95094079488,"internally_reserved":1531904,"allocated":95095611392,"data_stored":95095611392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1312037,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":13,"up_from":21,"seq":90194313227,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810640,"kb_used_data":92870828,"kb_used_omap":1019,"kb_used_meta":0,"kb_avail":92868976,"statfs":{"total":95927926784,"available":95097831424,"internally_reserved":1896448,"allocated":95099727872,"data_stored":95099727872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1043586,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":2,"up_from":21,"seq":90194313226,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":809212,"kb_used_data":92871276,"kb_used_omap":961,"kb_used_meta":0,"kb_avail":92870404,"statfs":{"total":95927926784,"available":95099293696,"internally_reserved":892928,"allocated":95100186624,"data_stored":95100186624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":984489,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]}]},{"osd":15,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810096,"kb_used_data":92870712,"kb_used_omap":902,"kb_used_meta":0,"kb_avail":92869520,"statfs":{"total":95927926784,"available":95098388480,"internally_reserved":1220608,"allocated":95099609088,"data_stored":95099609088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":924316,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":4,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":807472,"kb_used_data":92873552,"kb_used_omap":631,"kb_used_meta":0,"kb_avail":92872144,"statfs":{"total":95927926784,"available":95101075456,"internally_reserved":1441792,"allocated":95102517248,"data_stored":95102517248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":647020,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]}]},{"osd":5,"up_from":21,"seq":90194313226,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":806832,"kb_used_data":92873084,"kb_used_omap":660,"kb_used_meta":0,"kb_avail":92872784,"statfs":{"total":95927926784,"available":95101730816,"internally_reserved":307200,"allocated":95102038016,"data_stored":95102038016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":676743,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]}]},{"osd":6,"up_from":21,"seq":90194313226,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":811012,"kb_used_data":92870448,"kb_used_omap":814,"kb_used_meta":0,"kb_avail":92868604,"statfs":{"total":95927926784,"available":95097450496,"internally_reserved":1888256,"allocated":95099338752,"data_stored":95099338752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":833547,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":7,"up_from":21,"seq":90194313227,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":804700,"kb_used_data":92876296,"kb_used_omap":718,"kb_used_meta":0,"kb_avail":92874916,"statfs":{"total":95927926784,"available":95103913984,"internally_reserved":1413120,"allocated":95105327104,"data_stored":95105327104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":735673,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":8,"up_from":21,"seq":90194313226,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":809588,"kb_used_data":92870836,"kb_used_omap":860,"kb_used_meta":0,"kb_avail":92870028,"statfs":{"total":95927926784,"available":95098908672,"internally_reserved":827392,"allocated":95099736064,"data_stored":95099736064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":881511,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313226,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":806156,"kb_used_data":92874276,"kb_used_omap":924,"kb_used_meta":0,"kb_avail":92873460,"statfs":{"total":95927926784,"available":95102423040,"internally_reserved":835584,"allocated":95103258624,"data_stored":95103258624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":946313,"internal_metadata":0},"hb_peers":[0,1,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":488.03100000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]},{"osd":11,"up_from":21,"seq":90194313226,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":801468,"kb_used_data":92878376,"kb_used_omap":426,"kb_used_meta":0,"kb_avail":92878148,"statfs":{"total":95927926784,"available":95107223552,"internally_reserved":233472,"allocated":95107457024,"data_stored":95107457024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":436528,"internal_metadata":0},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":310.78199999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":12,"up_from":21,"seq":90194313227,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":802308,"kb_used_data":92878104,"kb_used_omap":481,"kb_used_meta":0,"kb_avail":92877308,"statfs":{"total":95927926784,"available":95106363392,"internally_reserved":815104,"allocated":95107178496,"data_stored":95107178496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":492790,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":1,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":809092,"kb_used_data":92871952,"kb_used_omap":1003,"kb_used_meta":0,"kb_avail":92870524,"statfs":{"total":95927926784,"available":95099416576,"internally_reserved":1462272,"allocated":95100878848,"data_stored":95100878848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1027244,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]}],"pool_statfs":[]}} 2024-09-15T01:32:57.400 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:32:57.439 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:57.438+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:57.494 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:57.494 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:32:57.494 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:32:57.494 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:32:57.494 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:57.503 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-15T01:32:57.540 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:57.538+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:57.609 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:32:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:32:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:32:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:32:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 34/31 ECSubWrite(tid=7, reqid=client.4477.0:44, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.416828795000001, 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.119995086, 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188439+0000", 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5613000000000001e-05 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307929+0000", 2024-09-15T01:32:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.119490443 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307936+0000", 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1740000000000003e-06 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308009+0000", 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2714999999999998e-05 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308018+0000", 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8019999999999998e-06 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308149+0000", 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130739 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308161+0000", 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2481e-05 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308199+0000", 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7492e-05 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308216+0000", 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7076999999999999e-05 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308418+0000", 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020204899999999999 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 39/38 ECSubWrite(tid=11, reqid=client.4477.0:196, at_version=39'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.398943580000001, 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042772959999999999, 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8800000000000002e-07 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206341+0000", 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2418e-05 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208467+0000", 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021264159999999999 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208477+0000", 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6090000000000007e-06 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208544+0000", 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6804000000000006e-05 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208555+0000", 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0869e-05 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208869+0000", 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031429800000000002 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208897+0000", 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8416999999999999e-05 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208959+0000", 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2201999999999998e-05 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208984+0000", 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4618999999999999e-05 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210585+0000", 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001601056 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 39/38 ECSubWrite(tid=12, reqid=client.4477.0:197, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.208410+0000", 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.396841135999999, 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042832929999999996, 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208410+0000", 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208410+0000", 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208411+0000", 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7100000000000011e-07 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208438+0000", 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6424e-05 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209653+0000", 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001215617 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209664+0000", 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0244999999999999e-05 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209722+0000", 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8675000000000003e-05 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209732+0000", 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3470000000000005e-06 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209848+0000", 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011628099999999999 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209861+0000", 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.359e-05 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209921+0000", 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9296000000000003e-05 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210211+0000", 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029074999999999999 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212694+0000", 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024820969999999999 2024-09-15T01:32:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 39/38 ECSubWrite(tid=13, reqid=client.4477.0:198, at_version=39'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.395676904999998, 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032744699999999998, 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:32:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2999999999999999e-07 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209623+0000", 2024-09-15T01:32:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7089999999999998e-05 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211317+0000", 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016948569999999999 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211325+0000", 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9699999999999999e-06 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211380+0000", 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4938000000000002e-05 2024-09-15T01:32:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211389+0000", 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7439999999999996e-06 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211510+0000", 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012121 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211545+0000", 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5209999999999997e-05 2024-09-15T01:32:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211601+0000", 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5541000000000002e-05 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211625+0000", 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4334000000000001e-05 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212849+0000", 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012236459999999999 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 39/31 ECSubWrite(tid=58, reqid=client.4477.0:240, at_version=39'20, trim_to=0'0, roll_forward_to=39'18))", 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.279710+0000", 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.325541406999999, 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029601300000000001, 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.279710+0000", 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.279710+0000", 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.279711+0000", 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.218e-06 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280721+0000", 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010096510000000001 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280766+0000", 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4734999999999998e-05 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280779+0000", 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3576e-05 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280836+0000", 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6864999999999999e-05 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280849+0000", 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3019e-05 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281118+0000", 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026896899999999998 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281128+0000", 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4450000000000002e-06 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281243+0000", 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115011 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281261+0000", 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8712999999999999e-05 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.282670+0000", 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001408928 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 39/38 ECSubWrite(tid=46, reqid=client.4477.0:247, at_version=39'9, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.298188+0000", 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.307063200999998, 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025692179999999998, 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298188+0000", 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298188+0000", 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298189+0000", 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.172e-06 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298425+0000", 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023576700000000001 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298454+0000", 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8711999999999999e-05 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298462+0000", 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4649999999999994e-06 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298576+0000", 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113979 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.298588+0000", 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1473e-05 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.299079+0000", 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049119500000000004 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.299089+0000", 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7229999999999999e-06 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.299215+0000", 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012652400000000001 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.299246+0000", 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0772000000000003e-05 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.300757+0000", 2024-09-15T01:32:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015114359999999999 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.241466+0000", 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.363784999, 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080416059999999998, 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241466+0000", 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241466+0000", 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241467+0000", 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3999999999999998e-07 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241497+0000", 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0596999999999999e-05 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243396+0000", 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018991819999999999 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243404+0000", 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2119999999999997e-06 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243529+0000", 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012554800000000001 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243541+0000", 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1834000000000001e-05 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243948+0000", 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040691300000000001 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244199+0000", 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025096899999999998 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244271+0000", 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1666000000000001e-05 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245015+0000", 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00074422900000000001 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249508+0000", 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044932160000000004 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.362763442, 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056362690000000002, 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0699999999999998e-07 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242511+0000", 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2926999999999999e-05 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242662+0000", 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015114499999999999 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242671+0000", 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7730000000000005e-06 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242736+0000", 2024-09-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4651000000000002e-05 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242746+0000", 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0203e-05 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242974+0000", 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022804800000000001 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243002+0000", 2024-09-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7792000000000001e-05 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243044+0000", 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2657999999999997e-05 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243365+0000", 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032055899999999999 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248124+0000", 2024-09-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047592060000000002 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242514+0000", 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.362737178, 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090428599999999998, 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242514+0000", 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242514+0000", 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242515+0000", 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242548+0000", 2024-09-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3615000000000003e-05 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242692+0000", 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014390900000000001 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242698+0000", 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4160000000000003e-06 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243566+0000", 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086795700000000004 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243574+0000", 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6889999999999993e-06 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243798+0000", 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000223611 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244198+0000", 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040037200000000002 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244270+0000", 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1956e-05 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244333+0000", 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2889e-05 2024-09-15T01:32:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251557+0000", 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072240569999999999 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 44/38 ECSubWrite(tid=20, reqid=osd.0.0:20, at_version=44'12, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.362685319000001, 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052822889999999999, 2024-09-15T01:32:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4600000000000001e-07 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242585+0000", 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9040000000000001e-05 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243433+0000", 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084765999999999997 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243441+0000", 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4179999999999997e-06 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243507+0000", 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5222999999999995e-05 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243517+0000", 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0567000000000001e-05 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243744+0000", 2024-09-15T01:32:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022733200000000001 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244198+0000", 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045302899999999998 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244269+0000", 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1445999999999993e-05 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244295+0000", 2024-09-15T01:32:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5799000000000002e-05 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247848+0000", 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003553629 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243616+0000", 2024-09-15T01:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.361635011000001, 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081799560000000004, 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243616+0000", 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243616+0000", 2024-09-15T01:32:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243617+0000", 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243646+0000", 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.879e-05 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243680+0000", 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4082000000000003e-05 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243686+0000", 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1589999999999998e-06 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243983+0000", 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029721700000000001 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243991+0000", 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6240000000000002e-06 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244172+0000", 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018103300000000001 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244199+0000", 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7254999999999999e-05 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244271+0000", 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.203e-05 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245044+0000", 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077298299999999998 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251796+0000", 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006752096 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.247492+0000", 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.357759602, 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045361799999999999, 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247492+0000", 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247492+0000", 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247494+0000", 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7230000000000001e-06 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247507+0000", 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3421000000000001e-05 2024-09-15T01:32:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247558+0000", 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0801999999999999e-05 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247567+0000", 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9889999999999998e-06 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247635+0000", 2024-09-15T01:32:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8623999999999999e-05 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247647+0000", 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1844999999999999e-05 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247812+0000", 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016474600000000001 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247865+0000", 2024-09-15T01:32:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3183000000000002e-05 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247909+0000", 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3645e-05 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247928+0000", 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9049e-05 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252028+0000", 2024-09-15T01:32:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041001529999999996 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 44/38 ECSubWrite(tid=21, reqid=osd.0.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.357025646, 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040969320000000002, 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:32:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248235+0000", 2024-09-15T01:32:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7180000000000002e-06 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248464+0000", 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022932900000000001 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248473+0000", 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.789e-06 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248516+0000", 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3380999999999997e-05 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248526+0000", 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2969999999999992e-06 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248624+0000", 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8672000000000007e-05 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248645+0000", 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0327e-05 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248697+0000", 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2602000000000002e-05 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248930+0000", 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023236900000000001 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252323+0000", 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003393063 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 44/38 ECSubWrite(tid=22, reqid=client.4477.0:293, at_version=44'14, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.271248+0000", 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.33400335, 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031070799999999999, 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271248+0000", 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271248+0000", 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271250+0000", 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.483e-06 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271392+0000", 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014198600000000001 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271464+0000", 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2108999999999997e-05 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271473+0000", 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2410000000000001e-06 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271641+0000", 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000167918 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271657+0000", 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6441000000000001e-05 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271939+0000", 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028217699999999999 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271952+0000", 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2741999999999999e-05 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272039+0000", 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6525000000000003e-05 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272070+0000", 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1443999999999999e-05 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.274355+0000", 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022850140000000001 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 44/31 ECSubWrite(tid=96, reqid=client.4477.0:321, at_version=44'36, trim_to=0'0, roll_forward_to=44'33))", 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.328792+0000", 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.276459062000001, 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026785049999999999, 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.328792+0000", 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.328792+0000", 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.328793+0000", 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-06 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329046+0000", 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000252978 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329077+0000", 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0784999999999997e-05 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329083+0000", 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0730000000000001e-06 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329127+0000", 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4230000000000002e-05 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329136+0000", 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6610000000000006e-06 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329438+0000", 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030188200000000001 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329447+0000", 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7669999999999992e-06 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329543+0000", 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5991999999999998e-05 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.329562+0000", 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8906000000000002e-05 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.331471+0000", 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001909191 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 54/38 ECSubWrite(tid=46, reqid=client.4477.0:453, at_version=54'27, trim_to=0'0, roll_forward_to=51'25))", 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.074963+0000", 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.530288093, 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030853130000000001, 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.074963+0000", 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.074963+0000", 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.074964+0000", 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-15T01:32:57.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.075000+0000", 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6043e-05 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.075119+0000", 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119214 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.075129+0000", 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.065e-05 2024-09-15T01:32:57.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.075210+0000", 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0679000000000005e-05 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.075221+0000", 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0448999999999999e-05 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.075497+0000", 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027638199999999998 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.075505+0000", 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8460000000000008e-06 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.075541+0000", 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6384000000000003e-05 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.075556+0000", 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5007999999999999e-05 2024-09-15T01:32:57.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.078049+0000", 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024924399999999998 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 55/31 ECSubWrite(tid=159, reqid=client.4477.0:511, at_version=55'52, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.075075+0000", 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.53017608999999999, 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080575170000000002, 2024-09-15T01:32:57.691 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075075+0000", 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075075+0000", 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075076+0000", 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9999999999999999e-07 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075102+0000", 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6071000000000002e-05 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075431+0000", 2024-09-15T01:32:57.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032964000000000002 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075438+0000", 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8349999999999998e-06 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075493+0000", 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5220999999999999e-05 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075504+0000", 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0434999999999999e-05 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075781+0000", 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027726299999999998 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075914+0000", 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132733 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075957+0000", 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3195999999999998e-05 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076225+0000", 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026822800000000002 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.083133+0000", 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069075949999999999 2024-09-15T01:32:57.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 55/31 ECSubWrite(tid=160, reqid=client.4477.0:512, at_version=55'53, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.075980+0000", 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.529271461, 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011476257, 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075980+0000", 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075980+0000", 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075981+0000", 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.54e-07 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075999+0000", 2024-09-15T01:32:57.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7974000000000001e-05 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076029+0000", 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0642000000000002e-05 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076035+0000", 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9750000000000004e-06 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076082+0000", 2024-09-15T01:32:57.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6227999999999999e-05 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076091+0000", 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2399999999999996e-06 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076185+0000", 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4315999999999996e-05 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076211+0000", 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5728000000000002e-05 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076263+0000", 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1900000000000001e-05 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076277+0000", 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4239999999999999e-05 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.087456+0000", 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01117916 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 55/31 ECSubWrite(tid=161, reqid=client.4477.0:513, at_version=55'54, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.076704+0000", 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.52854783800000005, 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010911452, 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076704+0000", 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076704+0000", 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076704+0000", 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076723+0000", 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8493000000000001e-05 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076754+0000", 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1479999999999997e-05 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076760+0000", 2024-09-15T01:32:57.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.105e-06 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076815+0000", 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5061e-05 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076824+0000", 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7110000000000001e-06 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076921+0000", 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7176999999999995e-05 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076950+0000", 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8606999999999999e-05 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.077617+0000", 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00066715999999999998 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.077636+0000", 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8779000000000001e-05 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.087615+0000", 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0099791570000000007 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 55/31 ECSubWrite(tid=162, reqid=client.4477.0:513, at_version=0'0, trim_to=0'0, roll_forward_to=55'54))", 2024-09-15T01:32:57.707 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.078911+0000", 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.52634066099999999, 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089080270000000007, 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078911+0000", 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078911+0000", 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:32:57.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078912+0000", 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.088e-06 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078924+0000", 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2592000000000001e-05 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078964+0000", 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9545999999999997e-05 2024-09-15T01:32:57.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078973+0000", 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7490000000000004e-06 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079047+0000", 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3755000000000001e-05 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079057+0000", 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0383e-05 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079195+0000", 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013842999999999999 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079269+0000", 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3774000000000002e-05 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079488+0000", 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021939400000000001 2024-09-15T01:32:57.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079506+0000", 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7153999999999999e-05 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.087819+0000", 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083131620000000007 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:32:57.713 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:32:57.713 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:32:57.714 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:32:57.714 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:57.713+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:57.723 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:57.724 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:32:57.735 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-09-15T01:32:57.076334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":55460403,"num_objects":54,"num_object_clones":9,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":1,"num_read":300,"num_read_kb":28806,"num_write":506,"num_write_kb":70274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":9879552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":607,"ondisk_log_size":607,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":15,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1405194240,"kb_used":12117760,"kb_used_data":1393092464,"kb_used_omap":12198,"kb_used_meta":0,"kb_avail":1393076480,"statfs":{"total":1438918901760,"available":1426510315520,"internally_reserved":16367616,"allocated":1426526683136,"data_stored":1426526683136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12491579,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":4,"commit_latency_ns":0,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5341184,"num_objects":-3,"num_object_clones":-4,"num_object_copies":-21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3,"num_whiteouts":1,"num_read":19,"num_read_kb":8633,"num_write":63,"num_write_kb":10448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5832704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000869"},"pg_stats":[{"pgid":"3.f","version":"52'70","reported_seq":137,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.159541+0000","last_change":"2024-09-15T01:32:54.083572+0000","last_active":"2024-09-15T01:32:54.159541+0000","last_peered":"2024-09-15T01:32:54.159541+0000","last_clean":"2024-09-15T01:32:54.159541+0000","last_became_active":"2024-09-15T01:32:34.487273+0000","last_became_peered":"2024-09-15T01:32:34.487273+0000","last_unstale":"2024-09-15T01:32:54.159541+0000","last_undegraded":"2024-09-15T01:32:54.159541+0000","last_fullsized":"2024-09-15T01:32:54.159541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:42.230488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072134800000000002,"stat_sum":{"num_bytes":4456448,"num_objects":7,"num_object_clones":2,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":23,"num_read_kb":149,"num_write":71,"num_write_kb":7776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"2"},{"start":"f","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.079870+0000","last_change":"2024-09-15T01:32:55.080194+0000","last_active":"2024-09-15T01:32:55.079870+0000","last_peered":"2024-09-15T01:32:55.079870+0000","last_clean":"2024-09-15T01:32:55.079870+0000","last_became_active":"2024-09-15T01:32:54.091710+0000","last_became_peered":"2024-09-15T01:32:54.091710+0000","last_unstale":"2024-09-15T01:32:55.079870+0000","last_undegraded":"2024-09-15T01:32:55.079870+0000","last_fullsized":"2024-09-15T01:32:55.079870+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:55.820742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067712999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,14,1,11,8,6,12],"acting":[2,14,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.d","version":"51'42","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.081416+0000","last_change":"2024-09-15T01:32:55.081742+0000","last_active":"2024-09-15T01:32:55.081416+0000","last_peered":"2024-09-15T01:32:55.081416+0000","last_clean":"2024-09-15T01:32:55.081416+0000","last_became_active":"2024-09-15T01:32:34.487040+0000","last_became_peered":"2024-09-15T01:32:34.487040+0000","last_unstale":"2024-09-15T01:32:55.081416+0000","last_undegraded":"2024-09-15T01:32:55.081416+0000","last_fullsized":"2024-09-15T01:32:55.081416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.966217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047890500000000001,"stat_sum":{"num_bytes":2473984,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":871,"num_write":19,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.c","version":"51'50","reported_seq":120,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.080667+0000","last_change":"2024-09-15T01:32:54.082408+0000","last_active":"2024-09-15T01:32:54.080667+0000","last_peered":"2024-09-15T01:32:54.080667+0000","last_clean":"2024-09-15T01:32:54.080667+0000","last_became_active":"2024-09-15T01:32:34.487072+0000","last_became_peered":"2024-09-15T01:32:34.487072+0000","last_unstale":"2024-09-15T01:32:54.080667+0000","last_undegraded":"2024-09-15T01:32:54.080667+0000","last_fullsized":"2024-09-15T01:32:54.080667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.186197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022585959999999999,"stat_sum":{"num_bytes":7864320,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":31,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"2"},{"start":"f","length":"1"}]},{"pgid":"3.b","version":"53'23","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.086294+0000","last_change":"2024-09-15T01:32:55.087737+0000","last_active":"2024-09-15T01:32:55.086294+0000","last_peered":"2024-09-15T01:32:55.086294+0000","last_clean":"2024-09-15T01:32:55.086294+0000","last_became_active":"2024-09-15T01:32:41.206211+0000","last_became_peered":"2024-09-15T01:32:41.206211+0000","last_unstale":"2024-09-15T01:32:55.086294+0000","last_undegraded":"2024-09-15T01:32:55.086294+0000","last_fullsized":"2024-09-15T01:32:55.086294+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.489551+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0087153419999999992,"stat_sum":{"num_bytes":2457600,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":10,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,13],"acting":[4,0,1,12,6,7,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.a","version":"44'19","reported_seq":123,"reported_epoch":53,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-15T01:32:55.086121+0000","last_change":"2024-09-15T01:32:55.082208+0000","last_active":"2024-09-15T01:32:53.073499+0000","last_peered":"2024-09-15T01:32:52.073534+0000","last_clean":"2024-09-15T01:32:52.073534+0000","last_became_active":"2024-09-15T01:32:41.201415+0000","last_became_peered":"2024-09-15T01:32:41.201415+0000","last_unstale":"2024-09-15T01:32:55.086121+0000","last_undegraded":"2024-09-15T01:32:55.080312+0000","last_fullsized":"2024-09-15T01:32:55.079754+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.689368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062814200000000004,"stat_sum":{"num_bytes":1654784,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":9,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8,14,6,11],"acting":[3,7,1,8,14,6,2147483647],"avail_no_missing":["3(0)","1(2)","6(5)","7(1)","8(3)","10(6)","14(4)"],"object_location_counts":[{"shards":"1(2),3(0),6(5),7(1),8(3),10(6),14(4)","objects":1}],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"1"}]},{"pgid":"3.9","version":"51'34","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.081224+0000","last_change":"2024-09-15T01:32:55.081698+0000","last_active":"2024-09-15T01:32:55.081224+0000","last_peered":"2024-09-15T01:32:55.081224+0000","last_clean":"2024-09-15T01:32:55.081224+0000","last_became_active":"2024-09-15T01:32:34.487117+0000","last_became_peered":"2024-09-15T01:32:34.487117+0000","last_unstale":"2024-09-15T01:32:55.081224+0000","last_undegraded":"2024-09-15T01:32:55.081224+0000","last_fullsized":"2024-09-15T01:32:55.081224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.760814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001249448,"stat_sum":{"num_bytes":4849664,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":2601,"num_write":19,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.8","version":"51'43","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.077556+0000","last_change":"2024-09-15T01:32:55.078177+0000","last_active":"2024-09-15T01:32:55.077556+0000","last_peered":"2024-09-15T01:32:55.077556+0000","last_clean":"2024-09-15T01:32:55.077556+0000","last_became_active":"2024-09-15T01:32:34.190371+0000","last_became_peered":"2024-09-15T01:32:34.190371+0000","last_unstale":"2024-09-15T01:32:55.077556+0000","last_undegraded":"2024-09-15T01:32:55.077556+0000","last_fullsized":"2024-09-15T01:32:55.077556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:51.142885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077278100000000005,"stat_sum":{"num_bytes":6258688,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":44,"num_write_kb":6112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"1.0","version":"24'112","reported_seq":173,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.077251+0000","last_change":"2024-09-15T01:32:16.749959+0000","last_active":"2024-09-15T01:32:54.077251+0000","last_peered":"2024-09-15T01:32:54.077251+0000","last_clean":"2024-09-15T01:32:54.077251+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:32:54.077251+0000","last_undegraded":"2024-09-15T01:32:54.077251+0000","last_fullsized":"2024-09-15T01:32:54.077251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_clean_scrub_stamp":"2024-09-15T01:32:15.451214+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.075494+0000","last_change":"2024-09-15T01:32:25.147954+0000","last_active":"2024-09-15T01:32:54.075494+0000","last_peered":"2024-09-15T01:32:54.075494+0000","last_clean":"2024-09-15T01:32:54.075494+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:32:54.075494+0000","last_undegraded":"2024-09-15T01:32:54.075494+0000","last_fullsized":"2024-09-15T01:32:54.075494+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.228776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"53'24","reported_seq":116,"reported_epoch":53,"state":"active+undersized+remapped","last_fresh":"2024-09-15T01:32:55.231718+0000","last_change":"2024-09-15T01:32:55.232233+0000","last_active":"2024-09-15T01:32:55.231718+0000","last_peered":"2024-09-15T01:32:55.231718+0000","last_clean":"2024-09-15T01:32:52.098439+0000","last_became_active":"2024-09-15T01:32:55.189455+0000","last_became_peered":"2024-09-15T01:32:55.189455+0000","last_unstale":"2024-09-15T01:32:55.231718+0000","last_undegraded":"2024-09-15T01:32:55.078442+0000","last_fullsized":"2024-09-15T01:32:55.077832+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:48.962933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0085917170000000005,"stat_sum":{"num_bytes":4341760,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":14,"num_read_kb":3846,"num_write":17,"num_write_kb":5776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":5652480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,2147483647],"avail_no_missing":["13(0)","1(5)","2(1)","3(2)","6(3)","11(6)","14(6)","15(4)"],"object_location_counts":[{"shards":"1(5),2(1),3(2),6(3),13(0),15(4)","objects":5}],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.076773+0000","last_change":"2024-09-15T01:32:25.143709+0000","last_active":"2024-09-15T01:32:54.076773+0000","last_peered":"2024-09-15T01:32:54.076773+0000","last_clean":"2024-09-15T01:32:54.076773+0000","last_became_active":"2024-09-15T01:32:23.134382+0000","last_became_peered":"2024-09-15T01:32:23.134382+0000","last_unstale":"2024-09-15T01:32:54.076773+0000","last_undegraded":"2024-09-15T01:32:54.076773+0000","last_fullsized":"2024-09-15T01:32:54.076773+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.326224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"53'8","reported_seq":126,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T01:32:55.207384+0000","last_change":"2024-09-15T01:32:55.205526+0000","last_active":"2024-09-15T01:32:55.207384+0000","last_peered":"2024-09-15T01:32:55.207384+0000","last_clean":"2024-09-15T01:32:52.072110+0000","last_became_active":"2024-09-15T01:32:55.189409+0000","last_became_peered":"2024-09-15T01:32:55.189409+0000","last_unstale":"2024-09-15T01:32:55.207384+0000","last_undegraded":"2024-09-15T01:32:55.079520+0000","last_fullsized":"2024-09-15T01:32:55.078828+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:47.882206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063910499999999995,"stat_sum":{"num_bytes":1867776,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1058,"num_write":8,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":442368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,15,5,10,2,13],"acting":[4,2147483647,15,5,10,2,13],"avail_no_missing":["4(0)","2(5)","5(3)","7(1)","10(4)","13(6)","15(2)"],"object_location_counts":[{"shards":"2(5),4(0),5(3),10(4),13(6),15(2)","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.077416+0000","last_change":"2024-09-15T01:32:25.583876+0000","last_active":"2024-09-15T01:32:54.077416+0000","last_peered":"2024-09-15T01:32:54.077416+0000","last_clean":"2024-09-15T01:32:54.077416+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:32:54.077416+0000","last_undegraded":"2024-09-15T01:32:54.077416+0000","last_fullsized":"2024-09-15T01:32:54.077416+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:49.092006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"53'34","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.085333+0000","last_change":"2024-09-15T01:32:55.085991+0000","last_active":"2024-09-15T01:32:55.085333+0000","last_peered":"2024-09-15T01:32:55.085333+0000","last_clean":"2024-09-15T01:32:55.085333+0000","last_became_active":"2024-09-15T01:32:34.190384+0000","last_became_peered":"2024-09-15T01:32:34.190384+0000","last_unstale":"2024-09-15T01:32:55.085333+0000","last_undegraded":"2024-09-15T01:32:55.085333+0000","last_fullsized":"2024-09-15T01:32:55.085333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.588055+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.004685486,"stat_sum":{"num_bytes":1703936,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":341,"num_write":14,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"2.2","version":"28'2","reported_seq":42,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.077457+0000","last_change":"2024-09-15T01:32:25.144113+0000","last_active":"2024-09-15T01:32:54.077457+0000","last_peered":"2024-09-15T01:32:54.077457+0000","last_clean":"2024-09-15T01:32:54.077457+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:32:54.077457+0000","last_undegraded":"2024-09-15T01:32:54.077457+0000","last_fullsized":"2024-09-15T01:32:54.077457+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"53'34","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.082017+0000","last_change":"2024-09-15T01:32:55.083145+0000","last_active":"2024-09-15T01:32:55.082017+0000","last_peered":"2024-09-15T01:32:55.082017+0000","last_clean":"2024-09-15T01:32:55.082017+0000","last_became_active":"2024-09-15T01:32:34.170218+0000","last_became_peered":"2024-09-15T01:32:34.170218+0000","last_unstale":"2024-09-15T01:32:55.082017+0000","last_undegraded":"2024-09-15T01:32:55.082017+0000","last_fullsized":"2024-09-15T01:32:55.082017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.093067+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0045204169999999997,"stat_sum":{"num_bytes":2867200,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":1544,"num_write":37,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.5","version":"52'24","reported_seq":97,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.180134+0000","last_change":"2024-09-15T01:32:54.083046+0000","last_active":"2024-09-15T01:32:54.180134+0000","last_peered":"2024-09-15T01:32:54.180134+0000","last_clean":"2024-09-15T01:32:54.180134+0000","last_became_active":"2024-09-15T01:32:34.189813+0000","last_became_peered":"2024-09-15T01:32:34.189813+0000","last_unstale":"2024-09-15T01:32:54.180134+0000","last_undegraded":"2024-09-15T01:32:54.180134+0000","last_fullsized":"2024-09-15T01:32:54.180134+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:47.814751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050507099999999995,"stat_sum":{"num_bytes":3637248,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":6006,"num_write":15,"num_write_kb":3936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"2"},{"start":"f","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.077209+0000","last_change":"2024-09-15T01:32:25.568260+0000","last_active":"2024-09-15T01:32:54.077209+0000","last_peered":"2024-09-15T01:32:54.077209+0000","last_clean":"2024-09-15T01:32:54.077209+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:32:54.077209+0000","last_undegraded":"2024-09-15T01:32:54.077209+0000","last_fullsized":"2024-09-15T01:32:54.077209+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:32.165806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"45'31","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.083986+0000","last_change":"2024-09-15T01:32:55.084335+0000","last_active":"2024-09-15T01:32:55.083986+0000","last_peered":"2024-09-15T01:32:55.083986+0000","last_clean":"2024-09-15T01:32:55.083986+0000","last_became_active":"2024-09-15T01:32:34.189644+0000","last_became_peered":"2024-09-15T01:32:34.189644+0000","last_unstale":"2024-09-15T01:32:55.083986+0000","last_undegraded":"2024-09-15T01:32:55.083986+0000","last_fullsized":"2024-09-15T01:32:55.083986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:02.903663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049015899999999999,"stat_sum":{"num_bytes":2359296,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":4567,"num_write":17,"num_write_kb":4528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.081132+0000","last_change":"2024-09-15T01:32:25.407396+0000","last_active":"2024-09-15T01:32:54.081132+0000","last_peered":"2024-09-15T01:32:54.081132+0000","last_clean":"2024-09-15T01:32:54.081132+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:32:54.081132+0000","last_undegraded":"2024-09-15T01:32:54.081132+0000","last_fullsized":"2024-09-15T01:32:54.081132+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:39.303667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"50'31","reported_seq":148,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T01:32:55.189951+0000","last_change":"2024-09-15T01:32:55.190604+0000","last_active":"2024-09-15T01:32:55.189951+0000","last_peered":"2024-09-15T01:32:55.189951+0000","last_clean":"2024-09-15T01:32:52.088276+0000","last_became_active":"2024-09-15T01:32:55.189564+0000","last_became_peered":"2024-09-15T01:32:55.189564+0000","last_unstale":"2024-09-15T01:32:55.189951+0000","last_undegraded":"2024-09-15T01:32:55.079592+0000","last_fullsized":"2024-09-15T01:32:55.078832+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:45.333547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015950892000000001,"stat_sum":{"num_bytes":5980160,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":917,"num_write":18,"num_write_kb":5840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,0,8,5,11],"acting":[4,6,1,0,8,5,2147483647],"avail_no_missing":["4(0)","0(3)","1(2)","5(5)","6(1)","8(4)","14(6)"],"object_location_counts":[{"shards":"0(3),1(2),4(0),5(5),6(1),8(4),14(6)","objects":7}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.080321+0000","last_change":"2024-09-15T01:32:25.568523+0000","last_active":"2024-09-15T01:32:54.080321+0000","last_peered":"2024-09-15T01:32:54.080321+0000","last_clean":"2024-09-15T01:32:54.080321+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:32:54.080321+0000","last_undegraded":"2024-09-15T01:32:54.080321+0000","last_fullsized":"2024-09-15T01:32:54.080321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"51'25","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.084087+0000","last_change":"2024-09-15T01:32:55.084545+0000","last_active":"2024-09-15T01:32:55.084087+0000","last_peered":"2024-09-15T01:32:55.084087+0000","last_clean":"2024-09-15T01:32:55.084087+0000","last_became_active":"2024-09-15T01:32:41.203815+0000","last_became_peered":"2024-09-15T01:32:41.203815+0000","last_unstale":"2024-09-15T01:32:55.084087+0000","last_undegraded":"2024-09-15T01:32:55.084087+0000","last_fullsized":"2024-09-15T01:32:55.084087+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:48.344390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066820499999999995,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":6811,"num_write":30,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,10],"acting":[0,13,1,15,12,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"2.7","version":"0'0","reported_seq":38,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T01:32:53.075475+0000","last_change":"2024-09-15T01:32:25.582726+0000","last_active":"2024-09-15T01:32:53.075475+0000","last_peered":"2024-09-15T01:32:53.075475+0000","last_clean":"2024-09-15T01:32:53.075475+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:32:53.075475+0000","last_undegraded":"2024-09-15T01:32:53.075475+0000","last_fullsized":"2024-09-15T01:32:53.075475+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.907962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":54870016,"num_objects":50,"num_object_clones":9,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":1,"num_read":214,"num_read_kb":28733,"num_write":359,"num_write_kb":67600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":9879552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":492,"ondisk_log_size":492,"up":112,"acting":112,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313228,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":811528,"kb_used_data":92868916,"kb_used_omap":1037,"kb_used_meta":0,"kb_avail":92868088,"statfs":{"total":95927926784,"available":95096922112,"internally_reserved":847872,"allocated":95097769984,"data_stored":95097769984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1062342,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":9,"up_from":0,"seq":90194313226,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":21,"seq":90194313227,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":803352,"kb_used_data":92877000,"kb_used_omap":476,"kb_used_meta":0,"kb_avail":92876264,"statfs":{"total":95927926784,"available":95105294336,"internally_reserved":753664,"allocated":95106048000,"data_stored":95106048000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":487440,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":0,"up_from":21,"seq":90194313228,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":814304,"kb_used_data":92866808,"kb_used_omap":1281,"kb_used_meta":0,"kb_avail":92865312,"statfs":{"total":95927926784,"available":95094079488,"internally_reserved":1531904,"allocated":95095611392,"data_stored":95095611392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1312037,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":13,"up_from":21,"seq":90194313227,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810640,"kb_used_data":92870828,"kb_used_omap":1019,"kb_used_meta":0,"kb_avail":92868976,"statfs":{"total":95927926784,"available":95097831424,"internally_reserved":1896448,"allocated":95099727872,"data_stored":95099727872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1043586,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":2,"up_from":21,"seq":90194313226,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":809212,"kb_used_data":92871276,"kb_used_omap":961,"kb_used_meta":0,"kb_avail":92870404,"statfs":{"total":95927926784,"available":95099293696,"internally_reserved":892928,"allocated":95100186624,"data_stored":95100186624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":984489,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]}]},{"osd":15,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810096,"kb_used_data":92870712,"kb_used_omap":902,"kb_used_meta":0,"kb_avail":92869520,"statfs":{"total":95927926784,"available":95098388480,"internally_reserved":1220608,"allocated":95099609088,"data_stored":95099609088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":924316,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":4,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":807472,"kb_used_data":92873552,"kb_used_omap":631,"kb_used_meta":0,"kb_avail":92872144,"statfs":{"total":95927926784,"available":95101075456,"internally_reserved":1441792,"allocated":95102517248,"data_stored":95102517248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":647020,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]}]},{"osd":5,"up_from":21,"seq":90194313226,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":806832,"kb_used_data":92873084,"kb_used_omap":660,"kb_used_meta":0,"kb_avail":92872784,"statfs":{"total":95927926784,"available":95101730816,"internally_reserved":307200,"allocated":95102038016,"data_stored":95102038016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":676743,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]}]},{"osd":6,"up_from":21,"seq":90194313226,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":811012,"kb_used_data":92870448,"kb_used_omap":814,"kb_used_meta":0,"kb_avail":92868604,"statfs":{"total":95927926784,"available":95097450496,"internally_reserved":1888256,"allocated":95099338752,"data_stored":95099338752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":833547,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":7,"up_from":21,"seq":90194313227,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":804700,"kb_used_data":92876296,"kb_used_omap":718,"kb_used_meta":0,"kb_avail":92874916,"statfs":{"total":95927926784,"available":95103913984,"internally_reserved":1413120,"allocated":95105327104,"data_stored":95105327104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":735673,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":8,"up_from":21,"seq":90194313226,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":809588,"kb_used_data":92870836,"kb_used_omap":860,"kb_used_meta":0,"kb_avail":92870028,"statfs":{"total":95927926784,"available":95098908672,"internally_reserved":827392,"allocated":95099736064,"data_stored":95099736064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":881511,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313226,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":806156,"kb_used_data":92874276,"kb_used_omap":924,"kb_used_meta":0,"kb_avail":92873460,"statfs":{"total":95927926784,"available":95102423040,"internally_reserved":835584,"allocated":95103258624,"data_stored":95103258624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":946313,"internal_metadata":0},"hb_peers":[0,1,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":488.03100000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]},{"osd":11,"up_from":21,"seq":90194313226,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":801468,"kb_used_data":92878376,"kb_used_omap":426,"kb_used_meta":0,"kb_avail":92878148,"statfs":{"total":95927926784,"available":95107223552,"internally_reserved":233472,"allocated":95107457024,"data_stored":95107457024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":436528,"internal_metadata":0},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":310.78199999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":12,"up_from":21,"seq":90194313227,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":802308,"kb_used_data":92878104,"kb_used_omap":481,"kb_used_meta":0,"kb_avail":92877308,"statfs":{"total":95927926784,"available":95106363392,"internally_reserved":815104,"allocated":95107178496,"data_stored":95107178496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":492790,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":1,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":809092,"kb_used_data":92871952,"kb_used_omap":1003,"kb_used_meta":0,"kb_avail":92870524,"statfs":{"total":95927926784,"available":95099416576,"internally_reserved":1462272,"allocated":95100878848,"data_stored":95100878848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1027244,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]}],"pool_statfs":[]}} 2024-09-15T01:32:57.737 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T01:32:57.815 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:32:57.814+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:32:57.822 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:32:57.822 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:57.831 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:32:57.892 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:32:57.915 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:57.914+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:32:57.936 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:32:58.016 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:58.015+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:58.018 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:32:58.019 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:32:58.039 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:32:58.039 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:32:58.040 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:58.040 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:58.049 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:32:58.058 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:32:58.059 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:32:58.071 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-15T01:32:58.071 INFO:tasks.rados.rados.0.smithi175.stdout:194: done (7 left) 2024-09-15T01:32:58.072 INFO:tasks.rados.rados.0.smithi175.stdout:195: delete oid 34 current snap is 11 2024-09-15T01:32:58.072 INFO:tasks.rados.rados.0.smithi175.stdout:191: finishing rollback tid 0 to smithi17529310-48 2024-09-15T01:32:58.072 INFO:tasks.rados.rados.0.smithi175.stdout:193: finishing rollback tid 0 to smithi17529310-18 2024-09-15T01:32:58.072 INFO:tasks.rados.rados.0.smithi175.stdout:191: finishing rollback tid 1 to smithi17529310-48 2024-09-15T01:32:58.072 INFO:tasks.rados.rados.0.smithi175.stdout:193: finishing rollback tid 1 to smithi17529310-18 2024-09-15T01:32:58.072 INFO:tasks.rados.rados.0.smithi175.stdout:191: finishing rollback tid 2 to smithi17529310-48 2024-09-15T01:32:58.072 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 48 v 54 (ObjNum 84 snap 7 seq_num 84) dirty exists 2024-09-15T01:32:58.072 INFO:tasks.rados.rados.0.smithi175.stdout:188: finishing copy_from racing read to smithi17529310-4 2024-09-15T01:32:58.072 INFO:tasks.rados.rados.0.smithi175.stdout:193: finishing rollback tid 2 to smithi17529310-18 2024-09-15T01:32:58.072 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 18 v 46 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-15T01:32:58.072 INFO:tasks.rados.rados.0.smithi175.stdout:192: finishing copy_from racing read to smithi17529310-49 2024-09-15T01:32:58.072 INFO:tasks.rados.rados.0.smithi175.stdout:189: finishing copy_from racing read to smithi17529310-7 2024-09-15T01:32:58.073 INFO:tasks.rados.rados.0.smithi175.stdout:192: finishing copy_from to smithi17529310-49 2024-09-15T01:32:58.073 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 49 v 39 (ObjNum 101 snap 10 seq_num 101) dirty exists 2024-09-15T01:32:58.073 INFO:tasks.rados.rados.0.smithi175.stdout:187: finishing copy_from racing read to smithi17529310-42 2024-09-15T01:32:58.073 INFO:tasks.rados.rados.0.smithi175.stdout:187: finishing copy_from to smithi17529310-42 2024-09-15T01:32:58.073 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 42 v 41 (ObjNum 105 snap 10 seq_num 105) dirty exists 2024-09-15T01:32:58.073 INFO:tasks.rados.rados.0.smithi175.stdout:188: finishing copy_from to smithi17529310-4 2024-09-15T01:32:58.073 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 4 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-15T01:32:58.073 INFO:tasks.rados.rados.0.smithi175.stdout:189: finishing copy_from to smithi17529310-7 2024-09-15T01:32:58.073 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 7 v 14 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-15T01:32:58.073 INFO:tasks.rados.rados.0.smithi175.stdout:190: finishing copy_from racing read to smithi17529310-45 2024-09-15T01:32:58.073 INFO:tasks.rados.rados.0.smithi175.stdout:190: finishing copy_from to smithi17529310-45 2024-09-15T01:32:58.073 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 45 v 70 (ObjNum 68 snap 3 seq_num 68) dirty exists 2024-09-15T01:32:58.081 INFO:tasks.rados.rados.0.smithi175.stdout:187: done (7 left) 2024-09-15T01:32:58.081 INFO:tasks.rados.rados.0.smithi175.stdout:188: done (6 left) 2024-09-15T01:32:58.081 INFO:tasks.rados.rados.0.smithi175.stdout:189: done (5 left) 2024-09-15T01:32:58.082 INFO:tasks.rados.rados.0.smithi175.stdout:190: done (4 left) 2024-09-15T01:32:58.082 INFO:tasks.rados.rados.0.smithi175.stdout:191: done (3 left) 2024-09-15T01:32:58.082 INFO:tasks.rados.rados.0.smithi175.stdout:192: done (2 left) 2024-09-15T01:32:58.082 INFO:tasks.rados.rados.0.smithi175.stdout:193: done (1 left) 2024-09-15T01:32:58.082 INFO:tasks.rados.rados.0.smithi175.stdout:195: done (0 left) 2024-09-15T01:32:58.082 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 17 current snap is 11 2024-09-15T01:32:58.082 INFO:tasks.rados.rados.0.smithi175.stdout:196: seq_num 107 ranges {311296=638976,950272=458752,1409024=704512} 2024-09-15T01:32:58.086 INFO:tasks.rados.rados.0.smithi175.stdout:196: writing smithi17529310-17 from 311296 to 950272 tid 1 2024-09-15T01:32:58.089 INFO:tasks.rados.rados.0.smithi175.stdout:196: writing smithi17529310-17 from 950272 to 1409024 tid 2 2024-09-15T01:32:58.090 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:32:58.090 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:32:58.094 INFO:tasks.rados.rados.0.smithi175.stdout:196: writing smithi17529310-17 from 1409024 to 2113536 tid 3 2024-09-15T01:32:58.094 INFO:tasks.rados.rados.0.smithi175.stdout:197: snap_create 2024-09-15T01:32:58.094 INFO:tasks.rados.rados.0.smithi175.stdout:196: finishing write tid 1 to smithi17529310-17 2024-09-15T01:32:58.097 INFO:tasks.rados.rados.0.smithi175.stdout:196: finishing write tid 2 to smithi17529310-17 2024-09-15T01:32:58.098 INFO:tasks.rados.rados.0.smithi175.stdout:196: finishing write tid 3 to smithi17529310-17 2024-09-15T01:32:58.109 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:32:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:32:58.116 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:58.115+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:58.128 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:32:58.128 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:32:58.130 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:58.130 INFO:teuthology.orchestra.run.smithi042.stdout:{"fsid":"d4803586-ed44-4154-8ee1-6b95ec3e17e7","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13/358 objects degraded (3.631%), 3 pgs degraded","count":3},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":50,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":56,"num_osds":16,"num_up_osds":16,"osd_up_since":1726363934,"num_in_osds":15,"osd_in_since":1726363957,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+undersized+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"activating+undersized+degraded+remapped","count":1}],"num_pgs":25,"num_pools":3,"num_objects":54,"data_bytes":55460403,"bytes_used":12408586240,"bytes_avail":1426510315520,"bytes_total":1438918901760,"inactive_pgs_ratio":0.039999999105930328,"degraded_objects":13,"degraded_total":358,"degraded_ratio":0.036312849162011177,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":1457859,"recovering_keys_per_sec":0,"num_objects_recovered":6,"num_bytes_recovered":5832704,"num_keys_recovered":0,"read_bytes_sec":2209567,"write_bytes_sec":2674106,"read_op_per_sec":4,"write_op_per_sec":15},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-15T01:32:07.286818+0000","services":{}},"progress_events":{}} 2024-09-15T01:32:58.143 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-15T01:32:58.143 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:32:58.148 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:32:58.148 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:32:58.168 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:32:58.168 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:32:58.186 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:32:58.187 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:32:58.187 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:58.187 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:58.196 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:32:58.206 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:32:58.206 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:32:58.210 INFO:tasks.rados.rados.0.smithi175.stdout:196: finishing write tid 4 to smithi17529310-17 2024-09-15T01:32:58.211 INFO:tasks.rados.rados.0.smithi175.stdout:196: finishing write tid 5 to smithi17529310-17 2024-09-15T01:32:58.211 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 17 v 93 (ObjNum 107 snap 11 seq_num 107) dirty exists 2024-09-15T01:32:58.211 INFO:tasks.rados.rados.0.smithi175.stdout:196: left oid 17 (ObjNum 107 snap 11 seq_num 107) 2024-09-15T01:32:58.217 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:32:58.216+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:32:58.224 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:32:58.225 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:32:58.243 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:32:58.243 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:32:58.262 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:32:58.263 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:32:58.282 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:32:58.282 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:32:58.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:32:58.286+0000 7fed51d9b640 0 -- 172.21.15.42:0/1151243177 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7fed54194ae0 msgr2=0x7fed54196f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:32:58.294 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:58.294 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:58.302 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:32:58.306 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:32:58.306 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:32:58.318 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:58.316+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:58.326 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:32:58.326 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:32:58.399 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:58.399 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:58.408 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:32:58.419 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:58.417+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:58.435 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:32:58.435 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:32:58.446 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-15T01:32:58.068280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":55460403,"num_objects":54,"num_object_clones":9,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":1,"num_read":300,"num_read_kb":28806,"num_write":506,"num_write_kb":70274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":9879552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":607,"ondisk_log_size":607,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":15,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1405194240,"kb_used":12117760,"kb_used_data":1393092464,"kb_used_omap":12198,"kb_used_meta":0,"kb_avail":1393076480,"statfs":{"total":1438918901760,"available":1426510315520,"internally_reserved":16367616,"allocated":1426526683136,"data_stored":1426526683136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12491579,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":4,"commit_latency_ns":0,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5341184,"num_objects":-3,"num_object_clones":-4,"num_object_copies":-21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3,"num_whiteouts":1,"num_read":19,"num_read_kb":8633,"num_write":63,"num_write_kb":10448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5832704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.993609"},"pg_stats":[{"pgid":"3.f","version":"52'70","reported_seq":137,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.159541+0000","last_change":"2024-09-15T01:32:54.083572+0000","last_active":"2024-09-15T01:32:54.159541+0000","last_peered":"2024-09-15T01:32:54.159541+0000","last_clean":"2024-09-15T01:32:54.159541+0000","last_became_active":"2024-09-15T01:32:34.487273+0000","last_became_peered":"2024-09-15T01:32:34.487273+0000","last_unstale":"2024-09-15T01:32:54.159541+0000","last_undegraded":"2024-09-15T01:32:54.159541+0000","last_fullsized":"2024-09-15T01:32:54.159541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:42.230488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072134800000000002,"stat_sum":{"num_bytes":4456448,"num_objects":7,"num_object_clones":2,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":23,"num_read_kb":149,"num_write":71,"num_write_kb":7776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"2"},{"start":"f","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.079870+0000","last_change":"2024-09-15T01:32:55.080194+0000","last_active":"2024-09-15T01:32:55.079870+0000","last_peered":"2024-09-15T01:32:55.079870+0000","last_clean":"2024-09-15T01:32:55.079870+0000","last_became_active":"2024-09-15T01:32:54.091710+0000","last_became_peered":"2024-09-15T01:32:54.091710+0000","last_unstale":"2024-09-15T01:32:55.079870+0000","last_undegraded":"2024-09-15T01:32:55.079870+0000","last_fullsized":"2024-09-15T01:32:55.079870+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:55.820742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067712999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,14,1,11,8,6,12],"acting":[2,14,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.d","version":"51'42","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.081416+0000","last_change":"2024-09-15T01:32:55.081742+0000","last_active":"2024-09-15T01:32:55.081416+0000","last_peered":"2024-09-15T01:32:55.081416+0000","last_clean":"2024-09-15T01:32:55.081416+0000","last_became_active":"2024-09-15T01:32:34.487040+0000","last_became_peered":"2024-09-15T01:32:34.487040+0000","last_unstale":"2024-09-15T01:32:55.081416+0000","last_undegraded":"2024-09-15T01:32:55.081416+0000","last_fullsized":"2024-09-15T01:32:55.081416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.966217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047890500000000001,"stat_sum":{"num_bytes":2473984,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":871,"num_write":19,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.c","version":"51'50","reported_seq":120,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.080667+0000","last_change":"2024-09-15T01:32:54.082408+0000","last_active":"2024-09-15T01:32:54.080667+0000","last_peered":"2024-09-15T01:32:54.080667+0000","last_clean":"2024-09-15T01:32:54.080667+0000","last_became_active":"2024-09-15T01:32:34.487072+0000","last_became_peered":"2024-09-15T01:32:34.487072+0000","last_unstale":"2024-09-15T01:32:54.080667+0000","last_undegraded":"2024-09-15T01:32:54.080667+0000","last_fullsized":"2024-09-15T01:32:54.080667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.186197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022585959999999999,"stat_sum":{"num_bytes":7864320,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":31,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"2"},{"start":"f","length":"1"}]},{"pgid":"3.b","version":"53'23","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.086294+0000","last_change":"2024-09-15T01:32:55.087737+0000","last_active":"2024-09-15T01:32:55.086294+0000","last_peered":"2024-09-15T01:32:55.086294+0000","last_clean":"2024-09-15T01:32:55.086294+0000","last_became_active":"2024-09-15T01:32:41.206211+0000","last_became_peered":"2024-09-15T01:32:41.206211+0000","last_unstale":"2024-09-15T01:32:55.086294+0000","last_undegraded":"2024-09-15T01:32:55.086294+0000","last_fullsized":"2024-09-15T01:32:55.086294+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.489551+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0087153419999999992,"stat_sum":{"num_bytes":2457600,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":10,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,13],"acting":[4,0,1,12,6,7,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.a","version":"44'19","reported_seq":123,"reported_epoch":53,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-15T01:32:55.086121+0000","last_change":"2024-09-15T01:32:55.082208+0000","last_active":"2024-09-15T01:32:53.073499+0000","last_peered":"2024-09-15T01:32:52.073534+0000","last_clean":"2024-09-15T01:32:52.073534+0000","last_became_active":"2024-09-15T01:32:41.201415+0000","last_became_peered":"2024-09-15T01:32:41.201415+0000","last_unstale":"2024-09-15T01:32:55.086121+0000","last_undegraded":"2024-09-15T01:32:55.080312+0000","last_fullsized":"2024-09-15T01:32:55.079754+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.689368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062814200000000004,"stat_sum":{"num_bytes":1654784,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":9,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8,14,6,11],"acting":[3,7,1,8,14,6,2147483647],"avail_no_missing":["3(0)","1(2)","6(5)","7(1)","8(3)","10(6)","14(4)"],"object_location_counts":[{"shards":"1(2),3(0),6(5),7(1),8(3),10(6),14(4)","objects":1}],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"1"}]},{"pgid":"3.9","version":"51'34","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.081224+0000","last_change":"2024-09-15T01:32:55.081698+0000","last_active":"2024-09-15T01:32:55.081224+0000","last_peered":"2024-09-15T01:32:55.081224+0000","last_clean":"2024-09-15T01:32:55.081224+0000","last_became_active":"2024-09-15T01:32:34.487117+0000","last_became_peered":"2024-09-15T01:32:34.487117+0000","last_unstale":"2024-09-15T01:32:55.081224+0000","last_undegraded":"2024-09-15T01:32:55.081224+0000","last_fullsized":"2024-09-15T01:32:55.081224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.760814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001249448,"stat_sum":{"num_bytes":4849664,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":2601,"num_write":19,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.8","version":"51'43","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.077556+0000","last_change":"2024-09-15T01:32:55.078177+0000","last_active":"2024-09-15T01:32:55.077556+0000","last_peered":"2024-09-15T01:32:55.077556+0000","last_clean":"2024-09-15T01:32:55.077556+0000","last_became_active":"2024-09-15T01:32:34.190371+0000","last_became_peered":"2024-09-15T01:32:34.190371+0000","last_unstale":"2024-09-15T01:32:55.077556+0000","last_undegraded":"2024-09-15T01:32:55.077556+0000","last_fullsized":"2024-09-15T01:32:55.077556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:51.142885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077278100000000005,"stat_sum":{"num_bytes":6258688,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":44,"num_write_kb":6112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"1.0","version":"24'112","reported_seq":173,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.077251+0000","last_change":"2024-09-15T01:32:16.749959+0000","last_active":"2024-09-15T01:32:54.077251+0000","last_peered":"2024-09-15T01:32:54.077251+0000","last_clean":"2024-09-15T01:32:54.077251+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:32:54.077251+0000","last_undegraded":"2024-09-15T01:32:54.077251+0000","last_fullsized":"2024-09-15T01:32:54.077251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_clean_scrub_stamp":"2024-09-15T01:32:15.451214+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.075494+0000","last_change":"2024-09-15T01:32:25.147954+0000","last_active":"2024-09-15T01:32:54.075494+0000","last_peered":"2024-09-15T01:32:54.075494+0000","last_clean":"2024-09-15T01:32:54.075494+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:32:54.075494+0000","last_undegraded":"2024-09-15T01:32:54.075494+0000","last_fullsized":"2024-09-15T01:32:54.075494+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.228776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"53'24","reported_seq":116,"reported_epoch":53,"state":"active+undersized+remapped","last_fresh":"2024-09-15T01:32:55.231718+0000","last_change":"2024-09-15T01:32:55.232233+0000","last_active":"2024-09-15T01:32:55.231718+0000","last_peered":"2024-09-15T01:32:55.231718+0000","last_clean":"2024-09-15T01:32:52.098439+0000","last_became_active":"2024-09-15T01:32:55.189455+0000","last_became_peered":"2024-09-15T01:32:55.189455+0000","last_unstale":"2024-09-15T01:32:55.231718+0000","last_undegraded":"2024-09-15T01:32:55.078442+0000","last_fullsized":"2024-09-15T01:32:55.077832+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:48.962933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0085917170000000005,"stat_sum":{"num_bytes":4341760,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":14,"num_read_kb":3846,"num_write":17,"num_write_kb":5776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":5652480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,2147483647],"avail_no_missing":["13(0)","1(5)","2(1)","3(2)","6(3)","11(6)","14(6)","15(4)"],"object_location_counts":[{"shards":"1(5),2(1),3(2),6(3),13(0),15(4)","objects":5}],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.076773+0000","last_change":"2024-09-15T01:32:25.143709+0000","last_active":"2024-09-15T01:32:54.076773+0000","last_peered":"2024-09-15T01:32:54.076773+0000","last_clean":"2024-09-15T01:32:54.076773+0000","last_became_active":"2024-09-15T01:32:23.134382+0000","last_became_peered":"2024-09-15T01:32:23.134382+0000","last_unstale":"2024-09-15T01:32:54.076773+0000","last_undegraded":"2024-09-15T01:32:54.076773+0000","last_fullsized":"2024-09-15T01:32:54.076773+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.326224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"53'8","reported_seq":126,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T01:32:55.207384+0000","last_change":"2024-09-15T01:32:55.205526+0000","last_active":"2024-09-15T01:32:55.207384+0000","last_peered":"2024-09-15T01:32:55.207384+0000","last_clean":"2024-09-15T01:32:52.072110+0000","last_became_active":"2024-09-15T01:32:55.189409+0000","last_became_peered":"2024-09-15T01:32:55.189409+0000","last_unstale":"2024-09-15T01:32:55.207384+0000","last_undegraded":"2024-09-15T01:32:55.079520+0000","last_fullsized":"2024-09-15T01:32:55.078828+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:47.882206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063910499999999995,"stat_sum":{"num_bytes":1867776,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1058,"num_write":8,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":442368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,15,5,10,2,13],"acting":[4,2147483647,15,5,10,2,13],"avail_no_missing":["4(0)","2(5)","5(3)","7(1)","10(4)","13(6)","15(2)"],"object_location_counts":[{"shards":"2(5),4(0),5(3),10(4),13(6),15(2)","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.077416+0000","last_change":"2024-09-15T01:32:25.583876+0000","last_active":"2024-09-15T01:32:54.077416+0000","last_peered":"2024-09-15T01:32:54.077416+0000","last_clean":"2024-09-15T01:32:54.077416+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:32:54.077416+0000","last_undegraded":"2024-09-15T01:32:54.077416+0000","last_fullsized":"2024-09-15T01:32:54.077416+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:49.092006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"53'34","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.085333+0000","last_change":"2024-09-15T01:32:55.085991+0000","last_active":"2024-09-15T01:32:55.085333+0000","last_peered":"2024-09-15T01:32:55.085333+0000","last_clean":"2024-09-15T01:32:55.085333+0000","last_became_active":"2024-09-15T01:32:34.190384+0000","last_became_peered":"2024-09-15T01:32:34.190384+0000","last_unstale":"2024-09-15T01:32:55.085333+0000","last_undegraded":"2024-09-15T01:32:55.085333+0000","last_fullsized":"2024-09-15T01:32:55.085333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.588055+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.004685486,"stat_sum":{"num_bytes":1703936,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":341,"num_write":14,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"2.2","version":"28'2","reported_seq":42,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.077457+0000","last_change":"2024-09-15T01:32:25.144113+0000","last_active":"2024-09-15T01:32:54.077457+0000","last_peered":"2024-09-15T01:32:54.077457+0000","last_clean":"2024-09-15T01:32:54.077457+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:32:54.077457+0000","last_undegraded":"2024-09-15T01:32:54.077457+0000","last_fullsized":"2024-09-15T01:32:54.077457+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"53'34","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.082017+0000","last_change":"2024-09-15T01:32:55.083145+0000","last_active":"2024-09-15T01:32:55.082017+0000","last_peered":"2024-09-15T01:32:55.082017+0000","last_clean":"2024-09-15T01:32:55.082017+0000","last_became_active":"2024-09-15T01:32:34.170218+0000","last_became_peered":"2024-09-15T01:32:34.170218+0000","last_unstale":"2024-09-15T01:32:55.082017+0000","last_undegraded":"2024-09-15T01:32:55.082017+0000","last_fullsized":"2024-09-15T01:32:55.082017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.093067+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0045204169999999997,"stat_sum":{"num_bytes":2867200,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":1544,"num_write":37,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"3.5","version":"52'24","reported_seq":97,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.180134+0000","last_change":"2024-09-15T01:32:54.083046+0000","last_active":"2024-09-15T01:32:54.180134+0000","last_peered":"2024-09-15T01:32:54.180134+0000","last_clean":"2024-09-15T01:32:54.180134+0000","last_became_active":"2024-09-15T01:32:34.189813+0000","last_became_peered":"2024-09-15T01:32:34.189813+0000","last_unstale":"2024-09-15T01:32:54.180134+0000","last_undegraded":"2024-09-15T01:32:54.180134+0000","last_fullsized":"2024-09-15T01:32:54.180134+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:47.814751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050507099999999995,"stat_sum":{"num_bytes":3637248,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":6006,"num_write":15,"num_write_kb":3936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"2"},{"start":"f","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.077209+0000","last_change":"2024-09-15T01:32:25.568260+0000","last_active":"2024-09-15T01:32:54.077209+0000","last_peered":"2024-09-15T01:32:54.077209+0000","last_clean":"2024-09-15T01:32:54.077209+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:32:54.077209+0000","last_undegraded":"2024-09-15T01:32:54.077209+0000","last_fullsized":"2024-09-15T01:32:54.077209+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:32.165806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"45'31","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.083986+0000","last_change":"2024-09-15T01:32:55.084335+0000","last_active":"2024-09-15T01:32:55.083986+0000","last_peered":"2024-09-15T01:32:55.083986+0000","last_clean":"2024-09-15T01:32:55.083986+0000","last_became_active":"2024-09-15T01:32:34.189644+0000","last_became_peered":"2024-09-15T01:32:34.189644+0000","last_unstale":"2024-09-15T01:32:55.083986+0000","last_undegraded":"2024-09-15T01:32:55.083986+0000","last_fullsized":"2024-09-15T01:32:55.083986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:02.903663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049015899999999999,"stat_sum":{"num_bytes":2359296,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":4567,"num_write":17,"num_write_kb":4528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.081132+0000","last_change":"2024-09-15T01:32:25.407396+0000","last_active":"2024-09-15T01:32:54.081132+0000","last_peered":"2024-09-15T01:32:54.081132+0000","last_clean":"2024-09-15T01:32:54.081132+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:32:54.081132+0000","last_undegraded":"2024-09-15T01:32:54.081132+0000","last_fullsized":"2024-09-15T01:32:54.081132+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:39.303667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"50'31","reported_seq":148,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T01:32:55.189951+0000","last_change":"2024-09-15T01:32:55.190604+0000","last_active":"2024-09-15T01:32:55.189951+0000","last_peered":"2024-09-15T01:32:55.189951+0000","last_clean":"2024-09-15T01:32:52.088276+0000","last_became_active":"2024-09-15T01:32:55.189564+0000","last_became_peered":"2024-09-15T01:32:55.189564+0000","last_unstale":"2024-09-15T01:32:55.189951+0000","last_undegraded":"2024-09-15T01:32:55.079592+0000","last_fullsized":"2024-09-15T01:32:55.078832+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:45.333547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015950892000000001,"stat_sum":{"num_bytes":5980160,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":917,"num_write":18,"num_write_kb":5840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,0,8,5,11],"acting":[4,6,1,0,8,5,2147483647],"avail_no_missing":["4(0)","0(3)","1(2)","5(5)","6(1)","8(4)","14(6)"],"object_location_counts":[{"shards":"0(3),1(2),4(0),5(5),6(1),8(4),14(6)","objects":7}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:32:54.080321+0000","last_change":"2024-09-15T01:32:25.568523+0000","last_active":"2024-09-15T01:32:54.080321+0000","last_peered":"2024-09-15T01:32:54.080321+0000","last_clean":"2024-09-15T01:32:54.080321+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:32:54.080321+0000","last_undegraded":"2024-09-15T01:32:54.080321+0000","last_fullsized":"2024-09-15T01:32:54.080321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"51'25","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:32:55.084087+0000","last_change":"2024-09-15T01:32:55.084545+0000","last_active":"2024-09-15T01:32:55.084087+0000","last_peered":"2024-09-15T01:32:55.084087+0000","last_clean":"2024-09-15T01:32:55.084087+0000","last_became_active":"2024-09-15T01:32:41.203815+0000","last_became_peered":"2024-09-15T01:32:41.203815+0000","last_unstale":"2024-09-15T01:32:55.084087+0000","last_undegraded":"2024-09-15T01:32:55.084087+0000","last_fullsized":"2024-09-15T01:32:55.084087+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:48.344390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066820499999999995,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":6811,"num_write":30,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,10],"acting":[0,13,1,15,12,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"2"},{"start":"c","length":"4"}]},{"pgid":"2.7","version":"0'0","reported_seq":38,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T01:32:53.075475+0000","last_change":"2024-09-15T01:32:25.582726+0000","last_active":"2024-09-15T01:32:53.075475+0000","last_peered":"2024-09-15T01:32:53.075475+0000","last_clean":"2024-09-15T01:32:53.075475+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:32:53.075475+0000","last_undegraded":"2024-09-15T01:32:53.075475+0000","last_fullsized":"2024-09-15T01:32:53.075475+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.907962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":54870016,"num_objects":50,"num_object_clones":9,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":1,"num_read":214,"num_read_kb":28733,"num_write":359,"num_write_kb":67600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":9879552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":492,"ondisk_log_size":492,"up":112,"acting":112,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313228,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":811528,"kb_used_data":92868916,"kb_used_omap":1037,"kb_used_meta":0,"kb_avail":92868088,"statfs":{"total":95927926784,"available":95096922112,"internally_reserved":847872,"allocated":95097769984,"data_stored":95097769984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1062342,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":9,"up_from":0,"seq":90194313226,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":21,"seq":90194313227,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":803352,"kb_used_data":92877000,"kb_used_omap":476,"kb_used_meta":0,"kb_avail":92876264,"statfs":{"total":95927926784,"available":95105294336,"internally_reserved":753664,"allocated":95106048000,"data_stored":95106048000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":487440,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":0,"up_from":21,"seq":90194313228,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":814304,"kb_used_data":92866808,"kb_used_omap":1281,"kb_used_meta":0,"kb_avail":92865312,"statfs":{"total":95927926784,"available":95094079488,"internally_reserved":1531904,"allocated":95095611392,"data_stored":95095611392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1312037,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":13,"up_from":21,"seq":90194313227,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810640,"kb_used_data":92870828,"kb_used_omap":1019,"kb_used_meta":0,"kb_avail":92868976,"statfs":{"total":95927926784,"available":95097831424,"internally_reserved":1896448,"allocated":95099727872,"data_stored":95099727872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1043586,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":2,"up_from":21,"seq":90194313226,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":809212,"kb_used_data":92871276,"kb_used_omap":961,"kb_used_meta":0,"kb_avail":92870404,"statfs":{"total":95927926784,"available":95099293696,"internally_reserved":892928,"allocated":95100186624,"data_stored":95100186624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":984489,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]}]},{"osd":15,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810096,"kb_used_data":92870712,"kb_used_omap":902,"kb_used_meta":0,"kb_avail":92869520,"statfs":{"total":95927926784,"available":95098388480,"internally_reserved":1220608,"allocated":95099609088,"data_stored":95099609088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":924316,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":4,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":807472,"kb_used_data":92873552,"kb_used_omap":631,"kb_used_meta":0,"kb_avail":92872144,"statfs":{"total":95927926784,"available":95101075456,"internally_reserved":1441792,"allocated":95102517248,"data_stored":95102517248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":647020,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]}]},{"osd":5,"up_from":21,"seq":90194313226,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":806832,"kb_used_data":92873084,"kb_used_omap":660,"kb_used_meta":0,"kb_avail":92872784,"statfs":{"total":95927926784,"available":95101730816,"internally_reserved":307200,"allocated":95102038016,"data_stored":95102038016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":676743,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]}]},{"osd":6,"up_from":21,"seq":90194313226,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":811012,"kb_used_data":92870448,"kb_used_omap":814,"kb_used_meta":0,"kb_avail":92868604,"statfs":{"total":95927926784,"available":95097450496,"internally_reserved":1888256,"allocated":95099338752,"data_stored":95099338752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":833547,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":7,"up_from":21,"seq":90194313227,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":804700,"kb_used_data":92876296,"kb_used_omap":718,"kb_used_meta":0,"kb_avail":92874916,"statfs":{"total":95927926784,"available":95103913984,"internally_reserved":1413120,"allocated":95105327104,"data_stored":95105327104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":735673,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":8,"up_from":21,"seq":90194313226,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":809588,"kb_used_data":92870836,"kb_used_omap":860,"kb_used_meta":0,"kb_avail":92870028,"statfs":{"total":95927926784,"available":95098908672,"internally_reserved":827392,"allocated":95099736064,"data_stored":95099736064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":881511,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313226,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":806156,"kb_used_data":92874276,"kb_used_omap":924,"kb_used_meta":0,"kb_avail":92873460,"statfs":{"total":95927926784,"available":95102423040,"internally_reserved":835584,"allocated":95103258624,"data_stored":95103258624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":946313,"internal_metadata":0},"hb_peers":[0,1,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":488.03100000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]},{"osd":11,"up_from":21,"seq":90194313226,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":801468,"kb_used_data":92878376,"kb_used_omap":426,"kb_used_meta":0,"kb_avail":92878148,"statfs":{"total":95927926784,"available":95107223552,"internally_reserved":233472,"allocated":95107457024,"data_stored":95107457024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":436528,"internal_metadata":0},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":310.78199999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":12,"up_from":21,"seq":90194313227,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":802308,"kb_used_data":92878104,"kb_used_omap":481,"kb_used_meta":0,"kb_avail":92877308,"statfs":{"total":95927926784,"available":95106363392,"internally_reserved":815104,"allocated":95107178496,"data_stored":95107178496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":492790,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":1,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":809092,"kb_used_data":92871952,"kb_used_omap":1003,"kb_used_meta":0,"kb_avail":92870524,"statfs":{"total":95927926784,"available":95099416576,"internally_reserved":1462272,"allocated":95100878848,"data_stored":95100878848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1027244,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]}],"pool_statfs":[]}} 2024-09-15T01:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:58.513 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:32:58.557 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:58.556+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:58.610 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:58.610 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:58.617 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:32:58.621 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:58.619+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:58.722 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:58.720+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:58.754 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:58.754 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:58.763 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-15T01:32:58.822 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:32:58.821+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:32:58.859 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:58.859 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:58.868 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-15T01:32:58.924 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:32:58.922+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:32:59.005 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:59.005 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:59.014 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-15T01:32:59.025 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:32:59.023+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:32:59.077 INFO:tasks.rados.rados.0.smithi175.stdout:196: done (1 left) 2024-09-15T01:32:59.078 INFO:tasks.rados.rados.0.smithi175.stdout:197: done (0 left) 2024-09-15T01:32:59.078 INFO:tasks.rados.rados.0.smithi175.stdout:198: copy_from oid 40 from oid 20 current snap is 12 2024-09-15T01:32:59.078 INFO:tasks.rados.rados.0.smithi175.stdout:199: delete oid 3 current snap is 12 2024-09-15T01:32:59.080 INFO:tasks.rados.rados.0.smithi175.stdout:199: done (1 left) 2024-09-15T01:32:59.081 INFO:tasks.rados.rados.0.smithi175.stdout:200: read oid 24 snap -1 2024-09-15T01:32:59.081 INFO:tasks.rados.rados.0.smithi175.stdout:200: expect deleted 2024-09-15T01:32:59.081 INFO:tasks.rados.rados.0.smithi175.stdout:201: rollback oid 28 current snap is 12 2024-09-15T01:32:59.081 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 28 to 10 2024-09-15T01:32:59.081 INFO:tasks.rados.rados.0.smithi175.stdout:202: snap_remove snap 11 2024-09-15T01:32:59.114 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:59.114 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:59.122 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:32:59.126 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:59.124+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:59.226 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:59.225+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:59.233 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:59.233 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:59.242 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:32:59.327 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:32:59.326+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:32:59.341 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:59.341 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:59.350 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:32:59.428 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:32:59.427+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:32:59.447 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:59.447 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:59.455 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:32:59.529 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:32:59.528+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:32:59.595 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:59.595 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:59.603 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:32:59.630 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:32:59.628+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:32:59.700 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:59.700 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:59.709 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:32:59.730 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:32:59.729+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:32:59.805 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:59.806 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:59.814 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:32:59.831 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:32:59.829+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:32:59.912 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:32:59.912 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:32:59.920 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:32:59.964 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:32:59.963+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:00.019 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:00.019 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:00.027 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:33:00.032 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:00.030+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:00.087 INFO:tasks.rados.rados.0.smithi175.stdout:201: finishing rollback tid 0 to smithi17529310-28 2024-09-15T01:33:00.087 INFO:tasks.rados.rados.0.smithi175.stdout:201: finishing rollback tid 1 to smithi17529310-28 2024-09-15T01:33:00.087 INFO:tasks.rados.rados.0.smithi175.stdout:201: finishing rollback tid 2 to smithi17529310-28 2024-09-15T01:33:00.087 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 28 v 45 (ObjNum 101 snap 10 seq_num 101) dirty exists 2024-09-15T01:33:00.088 INFO:tasks.rados.rados.0.smithi175.stdout:198: finishing copy_from racing read to smithi17529310-40 2024-09-15T01:33:00.088 INFO:tasks.rados.rados.0.smithi175.stdout:198: finishing copy_from to smithi17529310-40 2024-09-15T01:33:00.088 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 40 v 4 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-15T01:33:00.088 INFO:tasks.rados.rados.0.smithi175.stdout:198: done (3 left) 2024-09-15T01:33:00.088 INFO:tasks.rados.rados.0.smithi175.stdout:200: done (2 left) 2024-09-15T01:33:00.088 INFO:tasks.rados.rados.0.smithi175.stdout:201: done (1 left) 2024-09-15T01:33:00.088 INFO:tasks.rados.rados.0.smithi175.stdout:202: done (0 left) 2024-09-15T01:33:00.088 INFO:tasks.rados.rados.0.smithi175.stdout:203: rollback oid 44 current snap is 12 2024-09-15T01:33:00.088 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 44 to 10 2024-09-15T01:33:00.088 INFO:tasks.rados.rados.0.smithi175.stdout:204: delete oid 14 current snap is 12 2024-09-15T01:33:00.092 INFO:tasks.rados.rados.0.smithi175.stdout:203: finishing rollback tid 0 to smithi17529310-44 2024-09-15T01:33:00.095 INFO:tasks.rados.rados.0.smithi175.stdout:203: finishing rollback tid 1 to smithi17529310-44 2024-09-15T01:33:00.095 INFO:tasks.rados.rados.0.smithi175.stdout:203: finishing rollback tid 2 to smithi17529310-44 2024-09-15T01:33:00.095 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 44 v 49 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-15T01:33:00.095 INFO:tasks.rados.rados.0.smithi175.stdout:203: done (1 left) 2024-09-15T01:33:00.095 INFO:tasks.rados.rados.0.smithi175.stdout:204: done (0 left) 2024-09-15T01:33:00.095 INFO:tasks.rados.rados.0.smithi175.stdout:205: setattr oid 36 current snap is 12 2024-09-15T01:33:00.096 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:33:00.097 INFO:tasks.rados.rados.0.smithi175.stdout:206: read oid 7 snap -1 2024-09-15T01:33:00.097 INFO:tasks.rados.rados.0.smithi175.stdout:206: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-15T01:33:00.097 INFO:tasks.rados.rados.0.smithi175.stdout:207: read oid 12 snap 10 2024-09-15T01:33:00.097 INFO:tasks.rados.rados.0.smithi175.stdout:207: expect (ObjNum 98 snap 10 seq_num 98) 2024-09-15T01:33:00.097 INFO:tasks.rados.rados.0.smithi175.stdout:208: snap_create 2024-09-15T01:33:00.101 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 36 v 50 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-15T01:33:00.101 INFO:tasks.rados.rados.0.smithi175.stdout:206: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-15T01:33:00.112 INFO:tasks.rados.rados.0.smithi175.stdout:207: expect (ObjNum 98 snap 10 seq_num 98) 2024-09-15T01:33:00.132 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:00.131+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:00.170 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:00.170 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:00.179 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-15T01:33:00.234 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:00.232+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:00.277 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:00.277 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:00.287 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-15T01:33:00.335 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:00.332+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:00.337 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:33:00.385 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:00.385 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:00.393 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-15T01:33:00.436 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:00.434+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:00.465 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:00.484 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:00.484 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:00.490 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:00.490 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:00.498 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:33:00.504 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:00.504 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:00.524 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:00.537 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:00.535+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:00.542 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:00.542 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:00.561 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:00.581 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:00.600 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:00.600 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:00.605 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:00.605 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:00.605 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:00.605 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:00.614 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:33:00.618 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:00.618 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:00.637 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:00.637 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:00.638 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:00.636+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:00.656 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:00.656 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:00.676 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:00.694 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:00.712 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:00.713 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:00.713 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:00.713 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:00.721 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:33:00.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:00.731+0000 7f0c35224640 0 -- 172.21.15.42:0/561346630 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7f0c30194b70 msgr2=0x7f0c30196f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:00.739 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:00.758 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:00.758 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:00.764 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:00.763+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:00.820 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:00.820 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:00.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:00.820 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:00.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4302.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:23.147530+0000", 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 37.669762667000001, 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0086012669999999992, 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4302", 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.42:0/3794971532", 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 2 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147530+0000", 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147530+0000", 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147533+0000", 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.587e-06 2024-09-15T01:33:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147545+0000", 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1912999999999999e-05 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147569+0000", 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4389e-05 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147583+0000", 2024-09-15T01:33:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3913e-05 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147699+0000", 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116353 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147705+0000", 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7289999999999997e-06 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.155973+0000", 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0082681109999999999 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.156131+0000", 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015827199999999999 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:3 3.as0 3:5e9da7d3:::smithi17529310-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.174795+0000", 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 24.642497918, 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0075856939999999996, 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.174795+0000", 2024-09-15T01:33:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.174795+0000", 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.174795+0000", 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3799999999999997e-07 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.174797+0000", 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.818e-06 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.174801+0000", 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2089999999999999e-06 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.174804+0000", 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9720000000000001e-06 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176513+0000", 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001709052 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176585+0000", 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1571999999999998e-05 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.179928+0000", 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0033435130000000002 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.180018+0000", 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9316999999999999e-05 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.182381+0000", 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023627029999999999 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:192 3.as0 3:5e9da7d3:::smithi17529310-1:head [copy-from ver 6] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.624872022000002, 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01403071, 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 192 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192423+0000", 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3140000000000002e-06 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192431+0000", 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2320000000000001e-06 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192444+0000", 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3239e-05 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192454+0000", 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2439999999999999e-06 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192781+0000", 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032732699999999998 2024-09-15T01:33:00.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192904+0000", 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012317900000000001 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.201873+0000", 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089691839999999998 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202377+0000", 2024-09-15T01:33:00.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00050339000000000004 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202392+0000", 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5201999999999999e-05 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202869+0000", 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047669800000000001 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202888+0000", 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9202e-05 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203431+0000", 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00054287499999999995 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.206451+0000", 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003020624 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:193 3.as0 3:5e9da7d3:::smithi17529310-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192463+0000", 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.624829588000001, 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010399754000000001, 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 193 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192463+0000", 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192463+0000", 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192464+0000", 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192466+0000", 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5390000000000002e-06 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192476+0000", 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6549999999999999e-06 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192483+0000", 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9550000000000003e-06 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.193098+0000", 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00061465200000000004 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.193155+0000", 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7608999999999997e-05 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202537+0000", 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093813579999999994 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202652+0000", 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115473 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202863+0000", 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000210826 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 37/31 ECSubWrite(tid=20, reqid=client.4477.0:187, at_version=37'13, trim_to=0'0, roll_forward_to=34'10))", 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.194379+0000", 2024-09-15T01:33:00.841 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.622913275999998, 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0086632730000000008, 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194379+0000", 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194379+0000", 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194380+0000", 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7199999999999997e-07 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194404+0000", 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3587999999999998e-05 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194449+0000", 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4790999999999999e-05 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194457+0000", 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9640000000000003e-06 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194518+0000", 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1351000000000003e-05 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194528+0000", 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5999999999999996e-06 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194617+0000", 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9090000000000003e-05 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194627+0000", 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0029e-05 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194662+0000", 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5179000000000003e-05 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194679+0000", 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6816e-05 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203043+0000", 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0083638930000000007 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 37/31 ECSubWrite(tid=21, reqid=client.4477.0:188, at_version=37'14, trim_to=0'0, roll_forward_to=34'10))", 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.195155+0000", 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.622137510000002, 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0080205039999999995, 2024-09-15T01:33:00.846 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195155+0000", 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195155+0000", 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195156+0000", 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8299999999999996e-07 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195177+0000", 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1359999999999999e-05 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195213+0000", 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5194000000000002e-05 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195226+0000", 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3623e-05 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195265+0000", 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8532999999999999e-05 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195274+0000", 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2539999999999998e-06 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195386+0000", 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111734 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195397+0000", 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1589e-05 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.197123+0000", 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017253240000000001 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.197142+0000", 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9366000000000001e-05 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203176+0000", 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0060337439999999997 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:214 3.as0 3:5d5028c6:::smithi17529310-40:head [append 0~376832] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:41.208569+0000", 2024-09-15T01:33:00.851 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.608723994999998, 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0082088490000000007, 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 214 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208569+0000", 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208569+0000", 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208571+0000", 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1059999999999998e-06 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208873+0000", 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000301988 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208889+0000", 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6033999999999999e-05 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208896+0000", 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5630000000000003e-06 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208957+0000", 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0102999999999997e-05 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:00.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.209065+0000", 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108523 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.209750+0000", 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00068517899999999995 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.209839+0000", 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8289999999999997e-05 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210121+0000", 2024-09-15T01:33:00.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028203499999999998 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210219+0000", 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8450999999999996e-05 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210793+0000", 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00057359000000000002 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.216566+0000", 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005773172 2024-09-15T01:33:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.216778+0000", 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021181499999999999 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:215 3.as0 3:5d5028c6:::smithi17529310-40:head [setxattr _header (56) in=7b] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:41.208914+0000", 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.608379114000002, 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0081133950000000007, 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:00.857 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 215 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208914+0000", 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208914+0000", 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208915+0000", 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8999999999999995e-07 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208919+0000", 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2529999999999998e-06 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208926+0000", 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1250000000000004e-06 2024-09-15T01:33:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208929+0000", 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1549999999999999e-06 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.209920+0000", 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00099067899999999995 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.209999+0000", 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8782999999999994e-05 2024-09-15T01:33:00.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210390+0000", 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039110000000000002 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210491+0000", 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010117599999999999 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210501+0000", 2024-09-15T01:33:00.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0295000000000001e-05 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210677+0000", 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017592700000000001 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210794+0000", 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011722099999999999 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.216927+0000", 2024-09-15T01:33:00.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0061331770000000001 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.217027+0000", 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9613999999999999e-05 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:216 3.as0 3:5d5028c6:::smithi17529310-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:41.208948+0000", 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.608344978000002, 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0098072100000000002, 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.863 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 216 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208948+0000", 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208948+0000", 2024-09-15T01:33:00.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208948+0000", 2024-09-15T01:33:00.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999997e-07 2024-09-15T01:33:00.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208950+0000", 2024-09-15T01:33:00.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1710000000000001e-06 2024-09-15T01:33:00.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208958+0000", 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6599999999999995e-06 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208963+0000", 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4449999999999998e-06 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210679+0000", 2024-09-15T01:33:00.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017165189999999999 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210762+0000", 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2719000000000005e-05 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.217447+0000", 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0066855090000000001 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:00.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.217557+0000", 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011001 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.218755+0000", 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011976630000000001 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 44/31 ECSubWrite(tid=36, reqid=osd.7.0:36, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:00.868 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.245895+0000", 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.571398048000001, 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0067445059999999999, 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245895+0000", 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245895+0000", 2024-09-15T01:33:00.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245896+0000", 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.125e-06 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245943+0000", 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7172000000000002e-05 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246011+0000", 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8176999999999995e-05 2024-09-15T01:33:00.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246020+0000", 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0559999999999994e-06 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246086+0000", 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6048000000000002e-05 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246097+0000", 2024-09-15T01:33:00.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0654e-05 2024-09-15T01:33:00.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246955+0000", 2024-09-15T01:33:00.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085852599999999997 2024-09-15T01:33:00.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247002+0000", 2024-09-15T01:33:00.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6229000000000001e-05 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247047+0000", 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5597999999999999e-05 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247063+0000", 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5675000000000002e-05 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252639+0000", 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005576246 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.570317811000001, 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013669871, 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246976+0000", 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9800000000000002e-07 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247000+0000", 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4314000000000001e-05 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247045+0000", 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4938000000000002e-05 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247053+0000", 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247132+0000", 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8799000000000002e-05 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247142+0000", 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0511e-05 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247515+0000", 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037233299999999997 2024-09-15T01:33:00.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247546+0000", 2024-09-15T01:33:00.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1102000000000001e-05 2024-09-15T01:33:00.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247585+0000", 2024-09-15T01:33:00.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9152000000000001e-05 2024-09-15T01:33:00.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247603+0000", 2024-09-15T01:33:00.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-15T01:33:00.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.260645+0000", 2024-09-15T01:33:00.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013041662000000001 2024-09-15T01:33:00.879 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.879 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.879 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.880 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 44/31 ECSubWrite(tid=38, reqid=osd.7.0:37, at_version=0'0, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:00.880 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.251749+0000", 2024-09-15T01:33:00.880 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.565543550999999, 2024-09-15T01:33:00.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0092966969999999996, 2024-09-15T01:33:00.880 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.880 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.880 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251749+0000", 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251749+0000", 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251750+0000", 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6899999999999996e-07 2024-09-15T01:33:00.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251765+0000", 2024-09-15T01:33:00.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.489e-05 2024-09-15T01:33:00.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251796+0000", 2024-09-15T01:33:00.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0972000000000001e-05 2024-09-15T01:33:00.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251801+0000", 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3000000000000001e-06 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251845+0000", 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3900999999999999e-05 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251851+0000", 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3380000000000001e-06 2024-09-15T01:33:00.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251966+0000", 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011479700000000001 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252286+0000", 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031985800000000001 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252343+0000", 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6909e-05 2024-09-15T01:33:00.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252374+0000", 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0596000000000003e-05 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.261046+0000", 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0086723670000000003 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.885 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 48/31 ECSubWrite(tid=171, reqid=client.4477.0:402, at_version=48'24, trim_to=0'0, roll_forward_to=45'21))", 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:50.294480+0000", 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.522813096, 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0077438680000000001, 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294480+0000", 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294480+0000", 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294480+0000", 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3099999999999997e-07 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294818+0000", 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033747399999999999 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294863+0000", 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4987000000000001e-05 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294874+0000", 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1514e-05 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294927+0000", 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2629e-05 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294937+0000", 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0382e-05 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295264+0000", 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032685400000000001 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295274+0000", 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5510000000000005e-06 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295383+0000", 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010977800000000001 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295406+0000", 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2908000000000002e-05 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.302224+0000", 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0068171600000000001 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 48/31 ECSubWrite(tid=172, reqid=client.4477.0:403, at_version=48'25, trim_to=0'0, roll_forward_to=45'21))", 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:50.295531+0000", 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.52176188, 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0068974090000000002, 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295531+0000", 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295531+0000", 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295532+0000", 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295842+0000", 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031032899999999998 2024-09-15T01:33:00.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295879+0000", 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7038e-05 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295885+0000", 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.331e-06 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295965+0000", 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9587999999999998e-05 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295975+0000", 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0247e-05 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.296096+0000", 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012046 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.296106+0000", 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0380000000000001e-05 2024-09-15T01:33:00.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.296194+0000", 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8166999999999998e-05 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.296205+0000", 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0878e-05 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.302428+0000", 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0062232420000000004 2024-09-15T01:33:00.895 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=54, reqid=client.4477.0:516, at_version=55'44, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.074223+0000", 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.7430701759999998, 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089159619999999995, 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074223+0000", 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074223+0000", 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074224+0000", 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0860000000000001e-06 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074256+0000", 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2258000000000002e-05 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075603+0000", 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001347139 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075620+0000", 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7198999999999999e-05 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075708+0000", 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8177999999999995e-05 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075730+0000", 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1469000000000001e-05 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076062+0000", 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033251199999999998 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076077+0000", 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4523e-05 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076128+0000", 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1371e-05 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076153+0000", 2024-09-15T01:33:00.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4349e-05 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083139+0000", 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0069858780000000001 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=55, reqid=client.4477.0:517, at_version=55'45, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.741760304, 2024-09-15T01:33:00.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016315308000000001, 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075533+0000", 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9199999999999999e-07 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075557+0000", 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3264e-05 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076900+0000", 2024-09-15T01:33:00.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013436749999999999 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076916+0000", 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5302999999999999e-05 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076970+0000", 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4265999999999998e-05 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076980+0000", 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0521e-05 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077104+0000", 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012322599999999999 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077118+0000", 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4742e-05 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077164+0000", 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5972e-05 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077228+0000", 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3291000000000001e-05 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.091848+0000", 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014620055999999999 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.906 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=56, reqid=client.4477.0:518, at_version=55'46, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.740438669, 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015201957, 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076855+0000", 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.057e-06 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076876+0000", 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0874e-05 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079870+0000", 2024-09-15T01:33:00.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0029943259999999998 2024-09-15T01:33:00.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079882+0000", 2024-09-15T01:33:00.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2026e-05 2024-09-15T01:33:00.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079958+0000", 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5263000000000001e-05 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079969+0000", 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.145e-05 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.080116+0000", 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014736600000000001 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.080133+0000", 2024-09-15T01:33:00.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6776e-05 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082691+0000", 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025575229999999999 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082740+0000", 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9373000000000003e-05 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.092056+0000", 2024-09-15T01:33:00.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093159230000000003 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=57, reqid=client.4477.0:518, at_version=0'0, trim_to=0'0, roll_forward_to=55'46))", 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.7335149420000002, 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.77065714699999999, 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083779+0000", 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083792+0000", 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3478e-05 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853926+0000", 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.77013421000000004 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853932+0000", 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2820000000000004e-06 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853992+0000", 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9886999999999998e-05 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854001+0000", 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3700000000000001e-06 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854114+0000", 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000112651 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854152+0000", 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8518e-05 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854203+0000", 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1288000000000001e-05 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854223+0000", 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9884000000000001e-05 2024-09-15T01:33:00.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854435+0000", 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021157399999999999 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 56/54 ECSubWrite(tid=228, reqid=client.4477.0:520, at_version=56'33, trim_to=0'0, roll_forward_to=55'31))", 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.073896+0000", 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.7433971129999999, 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0079226000000000001, 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.917 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.073896+0000", 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.073896+0000", 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.073896+0000", 2024-09-15T01:33:00.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1199999999999999e-07 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.073929+0000", 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3324000000000001e-05 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074016+0000", 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7323000000000004e-05 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074024+0000", 2024-09-15T01:33:00.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5349999999999999e-06 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074093+0000", 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9442000000000004e-05 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074104+0000", 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0569e-05 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074386+0000", 2024-09-15T01:33:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028172599999999999 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074404+0000", 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8138000000000002e-05 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074454+0000", 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9929999999999998e-05 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074606+0000", 2024-09-15T01:33:00.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015201200000000001 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.081818+0000", 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0072123889999999996 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s3 56/31 ECSubWrite(tid=115, reqid=osd.14.0:115, at_version=56'45, trim_to=0'0, roll_forward_to=54'41))", 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.079178+0000", 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.738114983, 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093555189999999996, 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:00.922 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079178+0000", 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079178+0000", 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:00.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079179+0000", 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1299999999999998e-07 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079210+0000", 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1012000000000001e-05 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079243+0000", 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3686000000000002e-05 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079250+0000", 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5490000000000003e-06 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079312+0000", 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2397000000000006e-05 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079324+0000", 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1362999999999999e-05 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.081543+0000", 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022197779999999999 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082438+0000", 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00089462899999999998 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082489+0000", 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1326999999999998e-05 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082657+0000", 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016720600000000001 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.088533+0000", 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0058766590000000002 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:00.927 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:00.929 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:00.927+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:00.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:33:01.030 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:01.028+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:01.068 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:01.068 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:01.068 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:01.068 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:01.077 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:33:01.083 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-15T01:33:01.087 INFO:tasks.rados.rados.0.smithi175.stdout:205: done (3 left) 2024-09-15T01:33:01.087 INFO:tasks.rados.rados.0.smithi175.stdout:206: done (2 left) 2024-09-15T01:33:01.087 INFO:tasks.rados.rados.0.smithi175.stdout:207: done (1 left) 2024-09-15T01:33:01.087 INFO:tasks.rados.rados.0.smithi175.stdout:208: done (0 left) 2024-09-15T01:33:01.087 INFO:tasks.rados.rados.0.smithi175.stdout:209: rollback oid 42 current snap is 13 2024-09-15T01:33:01.087 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 42 to 12 2024-09-15T01:33:01.088 INFO:tasks.rados.rados.0.smithi175.stdout:210: setattr oid 46 current snap is 13 2024-09-15T01:33:01.088 INFO:tasks.rados.rados.0.smithi175.stdout:211: read oid 2 snap -1 2024-09-15T01:33:01.088 INFO:tasks.rados.rados.0.smithi175.stdout:211: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-15T01:33:01.088 INFO:tasks.rados.rados.0.smithi175.stdout:212: snap_create 2024-09-15T01:33:01.091 INFO:tasks.rados.rados.0.smithi175.stdout:209: finishing rollback tid 0 to smithi17529310-42 2024-09-15T01:33:01.091 INFO:tasks.rados.rados.0.smithi175.stdout:209: finishing rollback tid 1 to smithi17529310-42 2024-09-15T01:33:01.091 INFO:tasks.rados.rados.0.smithi175.stdout:209: finishing rollback tid 2 to smithi17529310-42 2024-09-15T01:33:01.091 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 42 v 59 (ObjNum 105 snap 10 seq_num 105) dirty exists 2024-09-15T01:33:01.094 INFO:tasks.rados.rados.0.smithi175.stdout:211: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-15T01:33:01.096 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 46 v 58 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-15T01:33:01.131 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:01.129+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:01.175 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:01.175 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:01.183 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:33:01.232 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:01.230+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:01.323 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:01.323 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:01.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:01.323 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:01.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.323 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:108 3.ds0 3:b063f44c:::smithi17529310-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:01.323 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:01.323 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.073088087999999, 2024-09-15T01:33:01.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013756292, 2024-09-15T01:33:01.323 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.324 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:01.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.324 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 108 2024-09-15T01:33:01.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.324 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:01.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:01.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246459+0000", 2024-09-15T01:33:01.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6800000000000001e-07 2024-09-15T01:33:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246461+0000", 2024-09-15T01:33:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.131e-06 2024-09-15T01:33:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246468+0000", 2024-09-15T01:33:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5529999999999998e-06 2024-09-15T01:33:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246475+0000", 2024-09-15T01:33:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5830000000000001e-06 2024-09-15T01:33:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.248249+0000", 2024-09-15T01:33:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017739990000000001 2024-09-15T01:33:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:01.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.248333+0000", 2024-09-15T01:33:01.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4288000000000003e-05 2024-09-15T01:33:01.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.249154+0000", 2024-09-15T01:33:01.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00082014900000000005 2024-09-15T01:33:01.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.249218+0000", 2024-09-15T01:33:01.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4519999999999999e-05 2024-09-15T01:33:01.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.260215+0000", 2024-09-15T01:33:01.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010996601 2024-09-15T01:33:01.329 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.329 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.329 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 34/31 ECSubWrite(tid=136, reqid=client.4477.0:154, at_version=34'9, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.308510+0000", 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.011036824000001, 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089133419999999994, 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308510+0000", 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308510+0000", 2024-09-15T01:33:01.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308510+0000", 2024-09-15T01:33:01.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-15T01:33:01.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308528+0000", 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7717e-05 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308567+0000", 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9118000000000001e-05 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308574+0000", 2024-09-15T01:33:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6089999999999997e-06 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308614+0000", 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0151999999999998e-05 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308623+0000", 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1509999999999993e-06 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308692+0000", 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9220000000000005e-05 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308729+0000", 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.676e-05 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308752+0000", 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2651e-05 2024-09-15T01:33:01.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.308767+0000", 2024-09-15T01:33:01.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5415000000000001e-05 2024-09-15T01:33:01.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.317423+0000", 2024-09-15T01:33:01.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0086558889999999999 2024-09-15T01:33:01.335 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.335 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:187 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [rollback 3] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192368+0000", 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.127179018, 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010685266000000001, 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.336 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 187 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192368+0000", 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192368+0000", 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192368+0000", 2024-09-15T01:33:01.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3300000000000002e-07 2024-09-15T01:33:01.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192372+0000", 2024-09-15T01:33:01.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.36e-06 2024-09-15T01:33:01.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192384+0000", 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2481e-05 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192390+0000", 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4810000000000003e-06 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.193649+0000", 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001258807 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.193722+0000", 2024-09-15T01:33:01.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3140999999999996e-05 2024-09-15T01:33:01.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194373+0000", 2024-09-15T01:33:01.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00065084699999999997 2024-09-15T01:33:01.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194463+0000", 2024-09-15T01:33:01.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9424000000000003e-05 2024-09-15T01:33:01.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194478+0000", 2024-09-15T01:33:01.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5693999999999999e-05 2024-09-15T01:33:01.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194534+0000", 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6008999999999999e-05 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194554+0000", 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9422000000000001e-05 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.196056+0000", 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001502506 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203053+0000", 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0069964609999999998 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:188 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [append 0~0] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192497+0000", 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.127049207999999, 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010740421999999999, 2024-09-15T01:33:01.343 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.344 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:01.344 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.344 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.344 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.344 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 188 2024-09-15T01:33:01.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.344 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192497+0000", 2024-09-15T01:33:01.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192497+0000", 2024-09-15T01:33:01.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192498+0000", 2024-09-15T01:33:01.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6899999999999997e-07 2024-09-15T01:33:01.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192502+0000", 2024-09-15T01:33:01.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.07e-06 2024-09-15T01:33:01.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192512+0000", 2024-09-15T01:33:01.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.039e-05 2024-09-15T01:33:01.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192517+0000", 2024-09-15T01:33:01.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7940000000000002e-06 2024-09-15T01:33:01.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194593+0000", 2024-09-15T01:33:01.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020753669999999998 2024-09-15T01:33:01.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194675+0000", 2024-09-15T01:33:01.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2756000000000005e-05 2024-09-15T01:33:01.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195151+0000", 2024-09-15T01:33:01.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047606299999999999 2024-09-15T01:33:01.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195243+0000", 2024-09-15T01:33:01.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1782999999999999e-05 2024-09-15T01:33:01.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195254+0000", 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0908e-05 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.197131+0000", 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001876401 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.197156+0000", 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5364999999999999e-05 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.197402+0000", 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024628300000000003 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203238+0000", 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0058355730000000001 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.350 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.078323585, 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014051065, 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:01.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:01.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:01.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241243+0000", 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0126e-05 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241422+0000", 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017906899999999999 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241429+0000", 2024-09-15T01:33:01.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4139999999999999e-06 2024-09-15T01:33:01.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241500+0000", 2024-09-15T01:33:01.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1098000000000003e-05 2024-09-15T01:33:01.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241510+0000", 2024-09-15T01:33:01.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8470000000000008e-06 2024-09-15T01:33:01.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241787+0000", 2024-09-15T01:33:01.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027729700000000003 2024-09-15T01:33:01.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242007+0000", 2024-09-15T01:33:01.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000220134 2024-09-15T01:33:01.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242146+0000", 2024-09-15T01:33:01.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013846799999999999 2024-09-15T01:33:01.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242278+0000", 2024-09-15T01:33:01.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0001326 2024-09-15T01:33:01.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.255274+0000", 2024-09-15T01:33:01.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01299575 2024-09-15T01:33:01.357 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.357 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.357 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.357 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.as1 44/38 ECSubWrite(tid=25, reqid=osd.3.0:25, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:01.357 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.241826+0000", 2024-09-15T01:33:01.357 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.077720586, 2024-09-15T01:33:01.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010494349, 2024-09-15T01:33:01.358 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.358 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.358 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241826+0000", 2024-09-15T01:33:01.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241826+0000", 2024-09-15T01:33:01.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241827+0000", 2024-09-15T01:33:01.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2200000000000002e-07 2024-09-15T01:33:01.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241852+0000", 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4672e-05 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241869+0000", 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7452000000000002e-05 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241876+0000", 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7390000000000002e-06 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242170+0000", 2024-09-15T01:33:01.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029464500000000001 2024-09-15T01:33:01.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242179+0000", 2024-09-15T01:33:01.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4500000000000004e-06 2024-09-15T01:33:01.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242421+0000", 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000242308 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242436+0000", 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4744e-05 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242608+0000", 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017215000000000001 2024-09-15T01:33:01.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242622+0000", 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4453999999999999e-05 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252320+0000", 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0096978129999999996 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.364 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s5 44/38 ECSubWrite(tid=20, reqid=osd.0.0:20, at_version=44'12, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:33:01.364 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.243644+0000", 2024-09-15T01:33:01.364 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.075902846, 2024-09-15T01:33:01.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012378171, 2024-09-15T01:33:01.364 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.364 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.364 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243644+0000", 2024-09-15T01:33:01.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243644+0000", 2024-09-15T01:33:01.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243644+0000", 2024-09-15T01:33:01.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-15T01:33:01.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243670+0000", 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6384999999999999e-05 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243811+0000", 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014044900000000001 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243817+0000", 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6680000000000004e-06 2024-09-15T01:33:01.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244023+0000", 2024-09-15T01:33:01.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020566099999999999 2024-09-15T01:33:01.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244033+0000", 2024-09-15T01:33:01.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9510000000000001e-06 2024-09-15T01:33:01.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245269+0000", 2024-09-15T01:33:01.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001236109 2024-09-15T01:33:01.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245301+0000", 2024-09-15T01:33:01.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1911e-05 2024-09-15T01:33:01.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245535+0000", 2024-09-15T01:33:01.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023357599999999999 2024-09-15T01:33:01.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245580+0000", 2024-09-15T01:33:01.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5327999999999997e-05 2024-09-15T01:33:01.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.256022+0000", 2024-09-15T01:33:01.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010441884 2024-09-15T01:33:01.370 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.370 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.370 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.370 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:01.370 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:01.370 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.075865240000001, 2024-09-15T01:33:01.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014188612, 2024-09-15T01:33:01.371 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.371 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.371 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:01.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:01.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243682+0000", 2024-09-15T01:33:01.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0299999999999999e-07 2024-09-15T01:33:01.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243699+0000", 2024-09-15T01:33:01.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7575999999999999e-05 2024-09-15T01:33:01.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244099+0000", 2024-09-15T01:33:01.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039909500000000003 2024-09-15T01:33:01.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244108+0000", 2024-09-15T01:33:01.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3209999999999994e-06 2024-09-15T01:33:01.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244168+0000", 2024-09-15T01:33:01.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9688999999999997e-05 2024-09-15T01:33:01.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244180+0000", 2024-09-15T01:33:01.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2031000000000001e-05 2024-09-15T01:33:01.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245504+0000", 2024-09-15T01:33:01.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013241520000000001 2024-09-15T01:33:01.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245537+0000", 2024-09-15T01:33:01.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3287000000000002e-05 2024-09-15T01:33:01.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245645+0000", 2024-09-15T01:33:01.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108435 2024-09-15T01:33:01.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245664+0000", 2024-09-15T01:33:01.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8329999999999999e-05 2024-09-15T01:33:01.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.257870+0000", 2024-09-15T01:33:01.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012206092999999999 2024-09-15T01:33:01.377 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.377 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.377 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.377 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 44/31 ECSubWrite(tid=152, reqid=osd.13.0:152, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:01.377 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.244552+0000", 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.074994278, 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010883719, 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244552+0000", 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244552+0000", 2024-09-15T01:33:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244552+0000", 2024-09-15T01:33:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8799999999999999e-07 2024-09-15T01:33:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244576+0000", 2024-09-15T01:33:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3506000000000002e-05 2024-09-15T01:33:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244641+0000", 2024-09-15T01:33:01.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5073000000000005e-05 2024-09-15T01:33:01.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244647+0000", 2024-09-15T01:33:01.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4160000000000001e-06 2024-09-15T01:33:01.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244702+0000", 2024-09-15T01:33:01.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4577999999999998e-05 2024-09-15T01:33:01.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244713+0000", 2024-09-15T01:33:01.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.084e-05 2024-09-15T01:33:01.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244938+0000", 2024-09-15T01:33:01.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022518700000000001 2024-09-15T01:33:01.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244968+0000", 2024-09-15T01:33:01.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9748999999999999e-05 2024-09-15T01:33:01.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245013+0000", 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5544000000000003e-05 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245032+0000", 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9091999999999999e-05 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.255436+0000", 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010403546 2024-09-15T01:33:01.383 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.384 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.384 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.384 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:01.384 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:01.384 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.072581657000001, 2024-09-15T01:33:01.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013797696999999999, 2024-09-15T01:33:01.385 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.385 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.385 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:01.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:01.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246966+0000", 2024-09-15T01:33:01.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000002e-07 2024-09-15T01:33:01.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246979+0000", 2024-09-15T01:33:01.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3111e-05 2024-09-15T01:33:01.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247017+0000", 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8096999999999998e-05 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247025+0000", 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9100000000000005e-06 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247095+0000", 2024-09-15T01:33:01.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0049000000000006e-05 2024-09-15T01:33:01.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247104+0000", 2024-09-15T01:33:01.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7919999999999998e-06 2024-09-15T01:33:01.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247217+0000", 2024-09-15T01:33:01.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011299 2024-09-15T01:33:01.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247376+0000", 2024-09-15T01:33:01.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015968800000000001 2024-09-15T01:33:01.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247417+0000", 2024-09-15T01:33:01.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0228999999999997e-05 2024-09-15T01:33:01.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248151+0000", 2024-09-15T01:33:01.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00073475999999999999 2024-09-15T01:33:01.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.260763+0000", 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012611282999999999 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.as1 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.247767+0000", 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.071779811000001, 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01354909, 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.392 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247767+0000", 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247767+0000", 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247767+0000", 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4299999999999999e-07 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247778+0000", 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1027e-05 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247838+0000", 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9571000000000003e-05 2024-09-15T01:33:01.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247842+0000", 2024-09-15T01:33:01.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9740000000000004e-06 2024-09-15T01:33:01.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247901+0000", 2024-09-15T01:33:01.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9694999999999997e-05 2024-09-15T01:33:01.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247923+0000", 2024-09-15T01:33:01.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1767e-05 2024-09-15T01:33:01.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248086+0000", 2024-09-15T01:33:01.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016259900000000001 2024-09-15T01:33:01.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248105+0000", 2024-09-15T01:33:01.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9609000000000001e-05 2024-09-15T01:33:01.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248157+0000", 2024-09-15T01:33:01.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1613999999999997e-05 2024-09-15T01:33:01.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248189+0000", 2024-09-15T01:33:01.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2140000000000001e-05 2024-09-15T01:33:01.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.261316+0000", 2024-09-15T01:33:01.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013126851 2024-09-15T01:33:01.398 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.398 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.398 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.398 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s5 44/38 ECSubWrite(tid=21, reqid=osd.0.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:01.398 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.248138+0000", 2024-09-15T01:33:01.399 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.071408979999999, 2024-09-15T01:33:01.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013472511, 2024-09-15T01:33:01.399 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.399 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.399 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248138+0000", 2024-09-15T01:33:01.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248138+0000", 2024-09-15T01:33:01.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248138+0000", 2024-09-15T01:33:01.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4600000000000001e-07 2024-09-15T01:33:01.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248143+0000", 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6489999999999996e-06 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248368+0000", 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022486699999999999 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248375+0000", 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0299999999999996e-06 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248429+0000", 2024-09-15T01:33:01.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3236999999999999e-05 2024-09-15T01:33:01.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248438+0000", 2024-09-15T01:33:01.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8060000000000006e-06 2024-09-15T01:33:01.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248547+0000", 2024-09-15T01:33:01.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108527 2024-09-15T01:33:01.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248566+0000", 2024-09-15T01:33:01.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9525999999999999e-05 2024-09-15T01:33:01.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248606+0000", 2024-09-15T01:33:01.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9730000000000001e-05 2024-09-15T01:33:01.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248621+0000", 2024-09-15T01:33:01.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4435e-05 2024-09-15T01:33:01.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.261610+0000", 2024-09-15T01:33:01.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012989458000000001 2024-09-15T01:33:01.405 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.405 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.405 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.405 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 44/31 ECSubWrite(tid=153, reqid=osd.13.0:152, at_version=0'0, trim_to=0'0, roll_forward_to=44'17))", 2024-09-15T01:33:01.405 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.248843+0000", 2024-09-15T01:33:01.405 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.070703117000001, 2024-09-15T01:33:01.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013295741, 2024-09-15T01:33:01.405 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.405 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.405 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248843+0000", 2024-09-15T01:33:01.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248843+0000", 2024-09-15T01:33:01.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248844+0000", 2024-09-15T01:33:01.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9599999999999999e-07 2024-09-15T01:33:01.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248853+0000", 2024-09-15T01:33:01.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.755e-06 2024-09-15T01:33:01.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248917+0000", 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3998999999999995e-05 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248923+0000", 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9139999999999996e-06 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248976+0000", 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3325000000000002e-05 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248986+0000", 2024-09-15T01:33:01.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9799999999999993e-06 2024-09-15T01:33:01.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.249081+0000", 2024-09-15T01:33:01.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5148000000000004e-05 2024-09-15T01:33:01.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.249121+0000", 2024-09-15T01:33:01.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0111000000000003e-05 2024-09-15T01:33:01.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.249145+0000", 2024-09-15T01:33:01.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4247999999999999e-05 2024-09-15T01:33:01.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.249158+0000", 2024-09-15T01:33:01.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3063e-05 2024-09-15T01:33:01.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.262139+0000", 2024-09-15T01:33:01.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012980702 2024-09-15T01:33:01.411 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.411 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.411 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.412 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:283 3.ds0 3:bfa066a3:::smithi17529310-39:head [copy-from ver 2] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:01.412 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:01.412 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.060780919000001, 2024-09-15T01:33:01.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015911307999999999, 2024-09-15T01:33:01.412 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.412 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:01.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.412 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 283 2024-09-15T01:33:01.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.412 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:01.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:01.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258768+0000", 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0980000000000001e-06 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258779+0000", 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1272e-05 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258795+0000", 2024-09-15T01:33:01.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5679999999999999e-05 2024-09-15T01:33:01.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258802+0000", 2024-09-15T01:33:01.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9029999999999998e-06 2024-09-15T01:33:01.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258879+0000", 2024-09-15T01:33:01.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7035999999999999e-05 2024-09-15T01:33:01.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.259139+0000", 2024-09-15T01:33:01.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260456 2024-09-15T01:33:01.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268247+0000", 2024-09-15T01:33:01.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0091076109999999998 2024-09-15T01:33:01.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268518+0000", 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027146499999999998 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268542+0000", 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3601e-05 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268616+0000", 2024-09-15T01:33:01.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3944999999999998e-05 2024-09-15T01:33:01.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.269087+0000", 2024-09-15T01:33:01.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047095399999999999 2024-09-15T01:33:01.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:01.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.274577+0000", 2024-09-15T01:33:01.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0054903019999999999 2024-09-15T01:33:01.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.274677+0000", 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9984999999999999e-05 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:284 3.ds0 3:bfa066a3:::smithi17529310-39:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.258815+0000", 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.060731537000001, 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010268249, 2024-09-15T01:33:01.419 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 284 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258815+0000", 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258815+0000", 2024-09-15T01:33:01.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258816+0000", 2024-09-15T01:33:01.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.82e-07 2024-09-15T01:33:01.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258818+0000", 2024-09-15T01:33:01.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3149999999999999e-06 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258826+0000", 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5620000000000004e-06 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258830+0000", 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.019e-06 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.259375+0000", 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00054464900000000002 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.259452+0000", 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6951000000000002e-05 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268607+0000", 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0091551800000000006 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268964+0000", 2024-09-15T01:33:01.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035634700000000002 2024-09-15T01:33:01.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.269083+0000", 2024-09-15T01:33:01.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011964399999999999 2024-09-15T01:33:01.424 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.424 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.425 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.425 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 44/31 ECSubWriteReply(tid=43, last_complete=44'32, committed=1, applied=1))", 2024-09-15T01:33:01.425 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:01.425 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.044100052999999, 2024-09-15T01:33:01.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.192594981, 2024-09-15T01:33:01.425 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.425 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.425 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275447+0000", 2024-09-15T01:33:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2099999999999995e-07 2024-09-15T01:33:01.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275452+0000", 2024-09-15T01:33:01.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2050000000000004e-06 2024-09-15T01:33:01.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.467889+0000", 2024-09-15T01:33:01.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19243722999999999 2024-09-15T01:33:01.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.467922+0000", 2024-09-15T01:33:01.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3237e-05 2024-09-15T01:33:01.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.468017+0000", 2024-09-15T01:33:01.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4543999999999995e-05 2024-09-15T01:33:01.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.468041+0000", 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4844000000000001e-05 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s5 54/38 ECSubWrite(tid=46, reqid=client.4477.0:453, at_version=54'27, trim_to=0'0, roll_forward_to=51'25))", 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:56.074729+0000", 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2448178060000004, 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0088713960000000001, 2024-09-15T01:33:01.429 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.430 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.430 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.074729+0000", 2024-09-15T01:33:01.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.074729+0000", 2024-09-15T01:33:01.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.074729+0000", 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.17e-07 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.074748+0000", 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9205999999999999e-05 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.074781+0000", 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3065000000000003e-05 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.074789+0000", 2024-09-15T01:33:01.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3409999999999997e-06 2024-09-15T01:33:01.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.074818+0000", 2024-09-15T01:33:01.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9448e-05 2024-09-15T01:33:01.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.074825+0000", 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.849e-06 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.074989+0000", 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016457 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.074994+0000", 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7650000000000001e-06 2024-09-15T01:33:01.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.075024+0000", 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0238000000000001e-05 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.075036+0000", 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1477e-05 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:56.083600+0000", 2024-09-15T01:33:01.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0085642200000000009 2024-09-15T01:33:01.437 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.437 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.437 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.437 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:517 3.ds0 3:b50d79c0:::smithi17529310-18:head [rollback 15] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:01.437 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.073290+0000", 2024-09-15T01:33:01.437 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.246256872, 2024-09-15T01:33:01.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0099547500000000001, 2024-09-15T01:33:01.437 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.438 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:01.438 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.438 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.438 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.438 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 517 2024-09-15T01:33:01.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.438 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073290+0000", 2024-09-15T01:33:01.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073290+0000", 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073290+0000", 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5300000000000003e-07 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073295+0000", 2024-09-15T01:33:01.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4070000000000003e-06 2024-09-15T01:33:01.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073304+0000", 2024-09-15T01:33:01.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7979999999999995e-06 2024-09-15T01:33:01.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073309+0000", 2024-09-15T01:33:01.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4700000000000001e-06 2024-09-15T01:33:01.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074718+0000", 2024-09-15T01:33:01.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001409071 2024-09-15T01:33:01.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074817+0000", 2024-09-15T01:33:01.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8472000000000002e-05 2024-09-15T01:33:01.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075579+0000", 2024-09-15T01:33:01.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00076189499999999996 2024-09-15T01:33:01.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075845+0000", 2024-09-15T01:33:01.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000266292 2024-09-15T01:33:01.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075865+0000", 2024-09-15T01:33:01.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9749999999999999e-05 2024-09-15T01:33:01.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075919+0000", 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4271999999999998e-05 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076793+0000", 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00087443899999999997 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:01.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.078347+0000", 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001554202 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083244+0000", 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0048969290000000004 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:518 3.ds0 3:b50d79c0:::smithi17529310-18:head [append 0~0] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:01.444 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.073314+0000", 2024-09-15T01:33:01.445 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.2462327330000003, 2024-09-15T01:33:01.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010662629999999999, 2024-09-15T01:33:01.445 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.445 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:01.445 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.445 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.445 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.445 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 518 2024-09-15T01:33:01.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.445 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073314+0000", 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073314+0000", 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073314+0000", 2024-09-15T01:33:01.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4000000000000002e-07 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073317+0000", 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3180000000000001e-06 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073322+0000", 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4380000000000003e-06 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073325+0000", 2024-09-15T01:33:01.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4829999999999999e-06 2024-09-15T01:33:01.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075985+0000", 2024-09-15T01:33:01.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026599309999999999 2024-09-15T01:33:01.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076083+0000", 2024-09-15T01:33:01.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7743000000000003e-05 2024-09-15T01:33:01.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076539+0000", 2024-09-15T01:33:01.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045620099999999999 2024-09-15T01:33:01.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076644+0000", 2024-09-15T01:33:01.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104972 2024-09-15T01:33:01.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076752+0000", 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107179 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076979+0000", 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000227347 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077620+0000", 2024-09-15T01:33:01.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00064142300000000001 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083336+0000", 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057156749999999999 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083976+0000", 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00064048000000000002 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.451 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.452 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.5s4 55/31 ECSubWrite(tid=49, reqid=client.4477.0:505, at_version=55'33, trim_to=0'0, roll_forward_to=54'31))", 2024-09-15T01:33:01.452 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.082609+0000", 2024-09-15T01:33:01.452 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.2369374799999999, 2024-09-15T01:33:01.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012942433999999999, 2024-09-15T01:33:01.452 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.452 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.452 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082609+0000", 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082609+0000", 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082609+0000", 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.17e-07 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082659+0000", 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0194000000000001e-05 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082708+0000", 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8046e-05 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082716+0000", 2024-09-15T01:33:01.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7269999999999996e-06 2024-09-15T01:33:01.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082772+0000", 2024-09-15T01:33:01.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5506e-05 2024-09-15T01:33:01.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082781+0000", 2024-09-15T01:33:01.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3549999999999994e-06 2024-09-15T01:33:01.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083044+0000", 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00026238900000000002 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083106+0000", 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2136000000000003e-05 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083184+0000", 2024-09-15T01:33:01.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8133000000000006e-05 2024-09-15T01:33:01.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083201+0000", 2024-09-15T01:33:01.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6884999999999998e-05 2024-09-15T01:33:01.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.095552+0000", 2024-09-15T01:33:01.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012350846 2024-09-15T01:33:01.457 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.458 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.458 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.458 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.458 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.458 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:01.458 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:33:01.459 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:33:01.459 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:01.388+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:01.491 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:01.489+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:01.556 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:01.557 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:01.557 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:01.557 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:01.565 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:33:01.592 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:01.590+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:01.661 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:01.661 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:01.669 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:33:01.693 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:01.691+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:01.754 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:33:01.755 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:33:01.767 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-15T01:33:01.080620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":71811635,"num_objects":58,"num_object_clones":12,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":3,"num_read":337,"num_read_kb":41893,"num_write":585,"num_write_kb":91346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":20021248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":15,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1405194240,"kb_used":12162128,"kb_used_data":1393053764,"kb_used_omap":15747,"kb_used_meta":0,"kb_avail":1393032112,"statfs":{"total":1438918901760,"available":1426464882688,"internally_reserved":22171648,"allocated":1426487054336,"data_stored":1426487054336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16125116,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":25,"commit_latency_ns":0,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16351232,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":37,"num_read_kb":13087,"num_write":79,"num_write_kb":21072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":10141696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.004286"},"pg_stats":[{"pgid":"3.f","version":"56'93","reported_seq":174,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T01:32:58.210671+0000","last_change":"2024-09-15T01:32:58.081834+0000","last_active":"2024-09-15T01:32:58.210671+0000","last_peered":"2024-09-15T01:32:58.210671+0000","last_clean":"2024-09-15T01:32:58.210671+0000","last_became_active":"2024-09-15T01:32:34.487273+0000","last_became_peered":"2024-09-15T01:32:34.487273+0000","last_unstale":"2024-09-15T01:32:58.210671+0000","last_undegraded":"2024-09-15T01:32:58.210671+0000","last_fullsized":"2024-09-15T01:32:58.210671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:42.230488+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0098134580000000006,"stat_sum":{"num_bytes":9715712,"num_objects":7,"num_object_clones":1,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":151,"num_write":82,"num_write_kb":12912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":113,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.086967+0000","last_change":"2024-09-15T01:33:00.087196+0000","last_active":"2024-09-15T01:33:00.086967+0000","last_peered":"2024-09-15T01:33:00.086967+0000","last_clean":"2024-09-15T01:33:00.086967+0000","last_became_active":"2024-09-15T01:32:54.091710+0000","last_became_peered":"2024-09-15T01:32:54.091710+0000","last_unstale":"2024-09-15T01:33:00.086967+0000","last_undegraded":"2024-09-15T01:33:00.086967+0000","last_fullsized":"2024-09-15T01:33:00.086967+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:55.820742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034944000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,14,1,11,8,6,12],"acting":[2,14,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"3.d","version":"58'50","reported_seq":137,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.088995+0000","last_change":"2024-09-15T01:33:00.088995+0000","last_active":"2024-09-15T01:33:00.088995+0000","last_peered":"2024-09-15T01:33:00.088995+0000","last_clean":"2024-09-15T01:33:00.088995+0000","last_became_active":"2024-09-15T01:32:34.487040+0000","last_became_peered":"2024-09-15T01:32:34.487040+0000","last_unstale":"2024-09-15T01:33:00.088995+0000","last_undegraded":"2024-09-15T01:33:00.088995+0000","last_fullsized":"2024-09-15T01:33:00.088995+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.966217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527575,"stat_sum":{"num_bytes":2473984,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":871,"num_write":41,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"3.c","version":"56'56","reported_seq":139,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T01:32:58.081582+0000","last_change":"2024-09-15T01:32:58.081582+0000","last_active":"2024-09-15T01:32:58.081582+0000","last_peered":"2024-09-15T01:32:58.081582+0000","last_clean":"2024-09-15T01:32:58.081582+0000","last_became_active":"2024-09-15T01:32:34.487072+0000","last_became_peered":"2024-09-15T01:32:34.487072+0000","last_unstale":"2024-09-15T01:32:58.081582+0000","last_undegraded":"2024-09-15T01:32:58.081582+0000","last_fullsized":"2024-09-15T01:32:58.081582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.186197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010697115,"stat_sum":{"num_bytes":7864320,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":33,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"}]},{"pgid":"3.b","version":"53'23","reported_seq":144,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.087513+0000","last_change":"2024-09-15T01:33:00.087840+0000","last_active":"2024-09-15T01:33:00.087513+0000","last_peered":"2024-09-15T01:33:00.087513+0000","last_clean":"2024-09-15T01:33:00.087513+0000","last_became_active":"2024-09-15T01:32:41.206211+0000","last_became_peered":"2024-09-15T01:32:41.206211+0000","last_unstale":"2024-09-15T01:33:00.087513+0000","last_undegraded":"2024-09-15T01:33:00.087513+0000","last_fullsized":"2024-09-15T01:33:00.087513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.489551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079285100000000002,"stat_sum":{"num_bytes":2457600,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":10,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,13],"acting":[4,0,1,12,6,7,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"3.a","version":"58'23","reported_seq":180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.091831+0000","last_change":"2024-09-15T01:33:00.093987+0000","last_active":"2024-09-15T01:33:00.091831+0000","last_peered":"2024-09-15T01:33:00.091831+0000","last_clean":"2024-09-15T01:33:00.091831+0000","last_became_active":"2024-09-15T01:32:57.079821+0000","last_became_peered":"2024-09-15T01:32:57.079821+0000","last_unstale":"2024-09-15T01:33:00.091831+0000","last_undegraded":"2024-09-15T01:33:00.091831+0000","last_fullsized":"2024-09-15T01:33:00.091831+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.988837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006026488,"stat_sum":{"num_bytes":1982464,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":10,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2359296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8,14,6,11],"acting":[3,7,1,8,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"3.9","version":"58'51","reported_seq":145,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.094046+0000","last_change":"2024-09-15T01:33:00.093392+0000","last_active":"2024-09-15T01:33:00.094046+0000","last_peered":"2024-09-15T01:33:00.094046+0000","last_clean":"2024-09-15T01:33:00.094046+0000","last_became_active":"2024-09-15T01:32:34.487117+0000","last_became_peered":"2024-09-15T01:32:34.487117+0000","last_unstale":"2024-09-15T01:33:00.094046+0000","last_undegraded":"2024-09-15T01:33:00.094046+0000","last_fullsized":"2024-09-15T01:33:00.094046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.760814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055311620000000001,"stat_sum":{"num_bytes":5046272,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":21,"num_read_kb":2602,"num_write":27,"num_write_kb":6416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"3.8","version":"56'55","reported_seq":146,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.101108+0000","last_change":"2024-09-15T01:33:00.085214+0000","last_active":"2024-09-15T01:33:00.101108+0000","last_peered":"2024-09-15T01:33:00.101108+0000","last_clean":"2024-09-15T01:33:00.101108+0000","last_became_active":"2024-09-15T01:32:34.190371+0000","last_became_peered":"2024-09-15T01:32:34.190371+0000","last_unstale":"2024-09-15T01:33:00.101108+0000","last_undegraded":"2024-09-15T01:33:00.101108+0000","last_fullsized":"2024-09-15T01:33:00.101108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:51.142885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068947699999999997,"stat_sum":{"num_bytes":11468800,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":5433,"num_write":50,"num_write_kb":11200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"1.0","version":"24'112","reported_seq":178,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.075322+0000","last_change":"2024-09-15T01:32:16.749959+0000","last_active":"2024-09-15T01:32:59.075322+0000","last_peered":"2024-09-15T01:32:59.075322+0000","last_clean":"2024-09-15T01:32:59.075322+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:32:59.075322+0000","last_undegraded":"2024-09-15T01:32:59.075322+0000","last_fullsized":"2024-09-15T01:32:59.075322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_clean_scrub_stamp":"2024-09-15T01:32:15.451214+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":45,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.075806+0000","last_change":"2024-09-15T01:32:25.147954+0000","last_active":"2024-09-15T01:32:59.075806+0000","last_peered":"2024-09-15T01:32:59.075806+0000","last_clean":"2024-09-15T01:32:59.075806+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:32:59.075806+0000","last_undegraded":"2024-09-15T01:32:59.075806+0000","last_fullsized":"2024-09-15T01:32:59.075806+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.228776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"56'33","reported_seq":163,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.084907+0000","last_change":"2024-09-15T01:33:00.085227+0000","last_active":"2024-09-15T01:33:00.084907+0000","last_peered":"2024-09-15T01:33:00.084907+0000","last_clean":"2024-09-15T01:33:00.084907+0000","last_became_active":"2024-09-15T01:32:57.985433+0000","last_became_peered":"2024-09-15T01:32:57.985433+0000","last_unstale":"2024-09-15T01:33:00.084907+0000","last_undegraded":"2024-09-15T01:33:00.084907+0000","last_fullsized":"2024-09-15T01:33:00.084907+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:55.388361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045281199999999997,"stat_sum":{"num_bytes":3719168,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":16,"num_read_kb":3846,"num_write":19,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":5652480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.078973+0000","last_change":"2024-09-15T01:32:25.143709+0000","last_active":"2024-09-15T01:32:59.078973+0000","last_peered":"2024-09-15T01:32:59.078973+0000","last_clean":"2024-09-15T01:32:59.078973+0000","last_became_active":"2024-09-15T01:32:23.134382+0000","last_became_peered":"2024-09-15T01:32:23.134382+0000","last_unstale":"2024-09-15T01:32:59.078973+0000","last_undegraded":"2024-09-15T01:32:59.078973+0000","last_fullsized":"2024-09-15T01:32:59.078973+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.326224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"53'8","reported_seq":170,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.087043+0000","last_change":"2024-09-15T01:33:00.087413+0000","last_active":"2024-09-15T01:33:00.087043+0000","last_peered":"2024-09-15T01:33:00.087043+0000","last_clean":"2024-09-15T01:33:00.087043+0000","last_became_active":"2024-09-15T01:32:58.074255+0000","last_became_peered":"2024-09-15T01:32:58.074255+0000","last_unstale":"2024-09-15T01:33:00.087043+0000","last_undegraded":"2024-09-15T01:33:00.087043+0000","last_fullsized":"2024-09-15T01:33:00.087043+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:51.030793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053781800000000002,"stat_sum":{"num_bytes":1867776,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1059,"num_write":8,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2129920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,15,5,10,2,13],"acting":[4,11,15,5,10,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.075326+0000","last_change":"2024-09-15T01:32:25.583876+0000","last_active":"2024-09-15T01:32:59.075326+0000","last_peered":"2024-09-15T01:32:59.075326+0000","last_clean":"2024-09-15T01:32:59.075326+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:32:59.075326+0000","last_undegraded":"2024-09-15T01:32:59.075326+0000","last_fullsized":"2024-09-15T01:32:59.075326+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:49.092006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"56'39","reported_seq":135,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.099298+0000","last_change":"2024-09-15T01:33:00.088424+0000","last_active":"2024-09-15T01:33:00.099298+0000","last_peered":"2024-09-15T01:33:00.099298+0000","last_clean":"2024-09-15T01:33:00.099298+0000","last_became_active":"2024-09-15T01:32:34.190384+0000","last_became_peered":"2024-09-15T01:32:34.190384+0000","last_unstale":"2024-09-15T01:33:00.099298+0000","last_undegraded":"2024-09-15T01:33:00.099298+0000","last_fullsized":"2024-09-15T01:33:00.099298+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.588055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027951300000000001,"stat_sum":{"num_bytes":819200,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3144,"num_write":16,"num_write_kb":2832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":47,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.079613+0000","last_change":"2024-09-15T01:32:25.144113+0000","last_active":"2024-09-15T01:32:59.079613+0000","last_peered":"2024-09-15T01:32:59.079613+0000","last_clean":"2024-09-15T01:32:59.079613+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:32:59.079613+0000","last_undegraded":"2024-09-15T01:32:59.079613+0000","last_fullsized":"2024-09-15T01:32:59.079613+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"58'49","reported_seq":143,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.088470+0000","last_change":"2024-09-15T01:33:00.089062+0000","last_active":"2024-09-15T01:33:00.088470+0000","last_peered":"2024-09-15T01:33:00.088470+0000","last_clean":"2024-09-15T01:33:00.088470+0000","last_became_active":"2024-09-15T01:32:34.170218+0000","last_became_peered":"2024-09-15T01:32:34.170218+0000","last_unstale":"2024-09-15T01:33:00.088470+0000","last_undegraded":"2024-09-15T01:33:00.088470+0000","last_fullsized":"2024-09-15T01:33:00.088470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.093067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040047889999999999,"stat_sum":{"num_bytes":4571136,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":21,"num_read_kb":1545,"num_write":45,"num_write_kb":5264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"3.5","version":"56'37","reported_seq":125,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T01:32:58.080172+0000","last_change":"2024-09-15T01:32:58.080172+0000","last_active":"2024-09-15T01:32:58.080172+0000","last_peered":"2024-09-15T01:32:58.080172+0000","last_clean":"2024-09-15T01:32:58.080172+0000","last_became_active":"2024-09-15T01:32:34.189813+0000","last_became_peered":"2024-09-15T01:32:34.189813+0000","last_unstale":"2024-09-15T01:32:58.080172+0000","last_undegraded":"2024-09-15T01:32:58.080172+0000","last_fullsized":"2024-09-15T01:32:58.080172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:47.814751+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0088917800000000002,"stat_sum":{"num_bytes":5455872,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":6008,"num_write":20,"num_write_kb":5712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.076663+0000","last_change":"2024-09-15T01:32:25.568260+0000","last_active":"2024-09-15T01:32:59.076663+0000","last_peered":"2024-09-15T01:32:59.076663+0000","last_clean":"2024-09-15T01:32:59.076663+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:32:59.076663+0000","last_undegraded":"2024-09-15T01:32:59.076663+0000","last_fullsized":"2024-09-15T01:32:59.076663+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:32.165806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"45'31","reported_seq":120,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.090790+0000","last_change":"2024-09-15T01:33:00.091439+0000","last_active":"2024-09-15T01:33:00.090790+0000","last_peered":"2024-09-15T01:33:00.090790+0000","last_clean":"2024-09-15T01:33:00.090790+0000","last_became_active":"2024-09-15T01:32:34.189644+0000","last_became_peered":"2024-09-15T01:32:34.189644+0000","last_unstale":"2024-09-15T01:33:00.090790+0000","last_undegraded":"2024-09-15T01:33:00.090790+0000","last_fullsized":"2024-09-15T01:33:00.090790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:02.903663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012088229999999999,"stat_sum":{"num_bytes":2359296,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":4567,"num_write":17,"num_write_kb":4528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.075219+0000","last_change":"2024-09-15T01:32:25.407396+0000","last_active":"2024-09-15T01:32:59.075219+0000","last_peered":"2024-09-15T01:32:59.075219+0000","last_clean":"2024-09-15T01:32:59.075219+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:32:59.075219+0000","last_undegraded":"2024-09-15T01:32:59.075219+0000","last_fullsized":"2024-09-15T01:32:59.075219+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:39.303667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"57'51","reported_seq":228,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.093428+0000","last_change":"2024-09-15T01:33:00.093787+0000","last_active":"2024-09-15T01:33:00.093428+0000","last_peered":"2024-09-15T01:33:00.093428+0000","last_clean":"2024-09-15T01:33:00.093428+0000","last_became_active":"2024-09-15T01:32:59.541051+0000","last_became_peered":"2024-09-15T01:32:59.541051+0000","last_unstale":"2024-09-15T01:33:00.093428+0000","last_undegraded":"2024-09-15T01:33:00.093428+0000","last_fullsized":"2024-09-15T01:33:00.093428+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:38.634992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006516651,"stat_sum":{"num_bytes":9322496,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":5768,"num_write":26,"num_write_kb":9264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":8208384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,0,8,5,11],"acting":[4,6,1,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.078898+0000","last_change":"2024-09-15T01:32:25.568523+0000","last_active":"2024-09-15T01:32:59.078898+0000","last_peered":"2024-09-15T01:32:59.078898+0000","last_clean":"2024-09-15T01:32:59.078898+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:32:59.078898+0000","last_undegraded":"2024-09-15T01:32:59.078898+0000","last_fullsized":"2024-09-15T01:32:59.078898+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"56'29","reported_seq":161,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.090690+0000","last_change":"2024-09-15T01:33:00.091140+0000","last_active":"2024-09-15T01:33:00.090690+0000","last_peered":"2024-09-15T01:33:00.090690+0000","last_clean":"2024-09-15T01:33:00.090690+0000","last_became_active":"2024-09-15T01:32:41.203815+0000","last_became_peered":"2024-09-15T01:32:41.203815+0000","last_unstale":"2024-09-15T01:33:00.090690+0000","last_undegraded":"2024-09-15T01:33:00.090690+0000","last_fullsized":"2024-09-15T01:33:00.090690+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:48.344390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060834599999999995,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":6811,"num_write":34,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,10],"acting":[0,13,1,15,12,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T01:32:58.071334+0000","last_change":"2024-09-15T01:32:25.582726+0000","last_active":"2024-09-15T01:32:58.071334+0000","last_peered":"2024-09-15T01:32:58.071334+0000","last_clean":"2024-09-15T01:32:58.071334+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:32:58.071334+0000","last_undegraded":"2024-09-15T01:32:58.071334+0000","last_fullsized":"2024-09-15T01:32:58.071334+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.907962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":71221248,"num_objects":54,"num_object_clones":12,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":3,"num_read":251,"num_read_kb":41820,"num_write":438,"num_write_kb":88672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":20021248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":628,"ondisk_log_size":628,"up":112,"acting":112,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313229,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810240,"kb_used_data":92871476,"kb_used_omap":1198,"kb_used_meta":0,"kb_avail":92869376,"statfs":{"total":95927926784,"available":95098241024,"internally_reserved":2150400,"allocated":95100391424,"data_stored":95100391424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1227693,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":3,"commit_latency_ns":0,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":574.26900000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":9,"up_from":0,"seq":90194313227,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":21,"seq":90194313228,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":803396,"kb_used_data":92876900,"kb_used_omap":639,"kb_used_meta":0,"kb_avail":92876220,"statfs":{"total":95927926784,"available":95105249280,"internally_reserved":696320,"allocated":95105945600,"data_stored":95105945600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":654916,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,8,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":3,"commit_latency_ns":0,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":0,"up_from":21,"seq":90194313229,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":820464,"kb_used_data":92861244,"kb_used_omap":1584,"kb_used_meta":0,"kb_avail":92859152,"statfs":{"total":95927926784,"available":95087771648,"internally_reserved":2142208,"allocated":95089913856,"data_stored":95089913856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1622327,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":13,"up_from":21,"seq":90194313228,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810652,"kb_used_data":92870592,"kb_used_omap":1350,"kb_used_meta":0,"kb_avail":92868964,"statfs":{"total":95927926784,"available":95097819136,"internally_reserved":1667072,"allocated":95099486208,"data_stored":95099486208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1383364,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":2,"up_from":21,"seq":90194313227,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":813364,"kb_used_data":92868836,"kb_used_omap":1210,"kb_used_meta":0,"kb_avail":92866252,"statfs":{"total":95927926784,"available":95095042048,"internally_reserved":2646016,"allocated":95097688064,"data_stored":95097688064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1239091,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":15,"up_from":21,"seq":90194313228,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810368,"kb_used_data":92870992,"kb_used_omap":1241,"kb_used_meta":0,"kb_avail":92869248,"statfs":{"total":95927926784,"available":95098109952,"internally_reserved":1785856,"allocated":95099895808,"data_stored":95099895808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1271807,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":4,"up_from":21,"seq":90194313228,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":807872,"kb_used_data":92872156,"kb_used_omap":844,"kb_used_meta":0,"kb_avail":92871744,"statfs":{"total":95927926784,"available":95100665856,"internally_reserved":421888,"allocated":95101087744,"data_stored":95101087744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":864564,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]}]},{"osd":5,"up_from":21,"seq":90194313227,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":813484,"kb_used_data":92867364,"kb_used_omap":858,"kb_used_meta":0,"kb_avail":92866132,"statfs":{"total":95927926784,"available":95094919168,"internally_reserved":1261568,"allocated":95096180736,"data_stored":95096180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":878683,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":6,"up_from":21,"seq":90194313227,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":814908,"kb_used_data":92866648,"kb_used_omap":1025,"kb_used_meta":0,"kb_avail":92864708,"statfs":{"total":95927926784,"available":95093460992,"internally_reserved":1986560,"allocated":95095447552,"data_stored":95095447552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1049737,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.353}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]}]},{"osd":7,"up_from":21,"seq":90194313228,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":802964,"kb_used_data":92876892,"kb_used_omap":953,"kb_used_meta":0,"kb_avail":92876652,"statfs":{"total":95927926784,"available":95105691648,"internally_reserved":245760,"allocated":95105937408,"data_stored":95105937408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":976577,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":8,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":814816,"kb_used_data":92866008,"kb_used_omap":1054,"kb_used_meta":0,"kb_avail":92864800,"statfs":{"total":95927926784,"available":95093555200,"internally_reserved":1236992,"allocated":95094792192,"data_stored":95094792192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1080139,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313227,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":811304,"kb_used_data":92869552,"kb_used_omap":1168,"kb_used_meta":0,"kb_avail":92868312,"statfs":{"total":95927926784,"available":95097151488,"internally_reserved":1269760,"allocated":95098421248,"data_stored":95098421248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1196891,"internal_metadata":0},"hb_peers":[0,1,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":11,"up_from":21,"seq":90194313227,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":813564,"kb_used_data":92868588,"kb_used_omap":706,"kb_used_meta":0,"kb_avail":92866052,"statfs":{"total":95927926784,"available":95094837248,"internally_reserved":2596864,"allocated":95097434112,"data_stored":95097434112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":723698,"internal_metadata":0},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":12,"up_from":21,"seq":90194313228,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":802340,"kb_used_data":92877380,"kb_used_omap":605,"kb_used_meta":0,"kb_avail":92877276,"statfs":{"total":95927926784,"available":95106330624,"internally_reserved":106496,"allocated":95106437120,"data_stored":95106437120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":619768,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":1,"up_from":21,"seq":90194313228,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":812392,"kb_used_data":92869136,"kb_used_omap":1304,"kb_used_meta":0,"kb_avail":92867224,"statfs":{"total":95927926784,"available":95096037376,"internally_reserved":1957888,"allocated":95097995264,"data_stored":95097995264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1335861,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]}]}],"pool_statfs":[]}} 2024-09-15T01:33:01.770 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T01:33:01.770 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:33:01.770 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-09-15T01:33:01.770 INFO:tasks.ceph.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 16 2024-09-15T01:33:01.771 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:33:01.772 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:01.772 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:01.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:01.772 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:01.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.773 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:50 3.fs0 3:f8e751c8:::smithi17529310-17:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:01.773 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.189642+0000", 2024-09-15T01:33:01.773 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.575793770000001, 2024-09-15T01:33:01.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035043355999999998, 2024-09-15T01:33:01.773 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.773 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:01.773 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.773 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.773 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.773 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 50 2024-09-15T01:33:01.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.774 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189642+0000", 2024-09-15T01:33:01.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189642+0000", 2024-09-15T01:33:01.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189643+0000", 2024-09-15T01:33:01.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7399999999999998e-07 2024-09-15T01:33:01.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189646+0000", 2024-09-15T01:33:01.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2009999999999999e-06 2024-09-15T01:33:01.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189652+0000", 2024-09-15T01:33:01.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0340000000000002e-06 2024-09-15T01:33:01.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189656+0000", 2024-09-15T01:33:01.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.481e-06 2024-09-15T01:33:01.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190946+0000", 2024-09-15T01:33:01.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0012900260000000001 2024-09-15T01:33:01.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191064+0000", 2024-09-15T01:33:01.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117945 2024-09-15T01:33:01.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191469+0000", 2024-09-15T01:33:01.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040500300000000003 2024-09-15T01:33:01.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191570+0000", 2024-09-15T01:33:01.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101067 2024-09-15T01:33:01.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191588+0000", 2024-09-15T01:33:01.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7844999999999999e-05 2024-09-15T01:33:01.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191633+0000", 2024-09-15T01:33:01.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5028000000000003e-05 2024-09-15T01:33:01.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191816+0000", 2024-09-15T01:33:01.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018250699999999999 2024-09-15T01:33:01.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:01.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.224605+0000", 2024-09-15T01:33:01.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032789051999999999 2024-09-15T01:33:01.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.224685+0000", 2024-09-15T01:33:01.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0692999999999995e-05 2024-09-15T01:33:01.781 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.781 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.781 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.781 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:51 3.fs0 3:f8e751c8:::smithi17529310-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:01.781 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.189675+0000", 2024-09-15T01:33:01.781 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.575760748, 2024-09-15T01:33:01.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.036634376000000003, 2024-09-15T01:33:01.782 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.782 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:01.782 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.782 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.782 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.782 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 51 2024-09-15T01:33:01.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.782 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189675+0000", 2024-09-15T01:33:01.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189675+0000", 2024-09-15T01:33:01.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189676+0000", 2024-09-15T01:33:01.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0900000000000002e-07 2024-09-15T01:33:01.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189678+0000", 2024-09-15T01:33:01.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9889999999999999e-06 2024-09-15T01:33:01.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189684+0000", 2024-09-15T01:33:01.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5309999999999998e-06 2024-09-15T01:33:01.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189688+0000", 2024-09-15T01:33:01.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6610000000000002e-06 2024-09-15T01:33:01.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191634+0000", 2024-09-15T01:33:01.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019467029999999999 2024-09-15T01:33:01.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191709+0000", 2024-09-15T01:33:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4474999999999994e-05 2024-09-15T01:33:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.224734+0000", 2024-09-15T01:33:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033024829999999998 2024-09-15T01:33:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.224781+0000", 2024-09-15T01:33:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6964e-05 2024-09-15T01:33:01.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.226309+0000", 2024-09-15T01:33:01.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001528714 2024-09-15T01:33:01.787 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.787 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.787 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.787 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:99 3.fs0 3:f630fe6e:::smithi17529310-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:01.787 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:01.788 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.530233349, 2024-09-15T01:33:01.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055995037999999997, 2024-09-15T01:33:01.788 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.788 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:01.788 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.788 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.788 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.788 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99 2024-09-15T01:33:01.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.788 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.788 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:01.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:01.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235203+0000", 2024-09-15T01:33:01.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4000000000000002e-07 2024-09-15T01:33:01.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235205+0000", 2024-09-15T01:33:01.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3e-06 2024-09-15T01:33:01.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235212+0000", 2024-09-15T01:33:01.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4389999999999997e-06 2024-09-15T01:33:01.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235216+0000", 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.031e-06 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.236877+0000", 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00166088 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:01.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.236933+0000", 2024-09-15T01:33:01.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6873999999999998e-05 2024-09-15T01:33:01.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255906+0000", 2024-09-15T01:33:01.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018972066999999999 2024-09-15T01:33:01.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255957+0000", 2024-09-15T01:33:01.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1467e-05 2024-09-15T01:33:01.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291198+0000", 2024-09-15T01:33:01.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035240540000000001 2024-09-15T01:33:01.793 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.793 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.793 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.793 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:113 3.fs0 3:f8072b57:::smithi17529310-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.253601+0000", 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.511834443000001, 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.037026959999999998, 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 113 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253601+0000", 2024-09-15T01:33:01.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253601+0000", 2024-09-15T01:33:01.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253602+0000", 2024-09-15T01:33:01.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8800000000000003e-07 2024-09-15T01:33:01.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253606+0000", 2024-09-15T01:33:01.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1930000000000003e-06 2024-09-15T01:33:01.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253617+0000", 2024-09-15T01:33:01.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1174e-05 2024-09-15T01:33:01.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253621+0000", 2024-09-15T01:33:01.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6009999999999999e-06 2024-09-15T01:33:01.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.254591+0000", 2024-09-15T01:33:01.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000970114 2024-09-15T01:33:01.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.254714+0000", 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012272300000000001 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255159+0000", 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044522399999999998 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255354+0000", 2024-09-15T01:33:01.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000194666 2024-09-15T01:33:01.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255360+0000", 2024-09-15T01:33:01.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5359999999999998e-06 2024-09-15T01:33:01.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255397+0000", 2024-09-15T01:33:01.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6613000000000003e-05 2024-09-15T01:33:01.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255520+0000", 2024-09-15T01:33:01.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123161 2024-09-15T01:33:01.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:01.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290561+0000", 2024-09-15T01:33:01.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035041471999999997 2024-09-15T01:33:01.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290628+0000", 2024-09-15T01:33:01.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6995000000000005e-05 2024-09-15T01:33:01.801 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.801 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.801 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.801 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:114 3.fs0 3:f8072b57:::smithi17529310-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:01.801 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:01.801 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.511793428000001, 2024-09-15T01:33:01.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048699039999999999, 2024-09-15T01:33:01.802 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.802 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:01.802 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.802 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.802 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.802 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 114 2024-09-15T01:33:01.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.802 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:01.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:01.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253643+0000", 2024-09-15T01:33:01.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6899999999999998e-07 2024-09-15T01:33:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253646+0000", 2024-09-15T01:33:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-15T01:33:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253652+0000", 2024-09-15T01:33:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3369999999999996e-06 2024-09-15T01:33:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253656+0000", 2024-09-15T01:33:01.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.031e-06 2024-09-15T01:33:01.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255410+0000", 2024-09-15T01:33:01.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017543420000000001 2024-09-15T01:33:01.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:01.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255505+0000", 2024-09-15T01:33:01.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4704999999999995e-05 2024-09-15T01:33:01.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290833+0000", 2024-09-15T01:33:01.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035327711999999997 2024-09-15T01:33:01.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290893+0000", 2024-09-15T01:33:01.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0384999999999999e-05 2024-09-15T01:33:01.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.302341+0000", 2024-09-15T01:33:01.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011448274 2024-09-15T01:33:01.807 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.807 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.807 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.808 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpReadReply(3.fs0 34/31 ECSubReadReply(tid=23, attrs_read=0))", 2024-09-15T01:33:01.808 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.256270+0000", 2024-09-15T01:33:01.808 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.509165795000001, 2024-09-15T01:33:01.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034934552000000001, 2024-09-15T01:33:01.808 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.808 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.808 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.256270+0000", 2024-09-15T01:33:01.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.256270+0000", 2024-09-15T01:33:01.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.256270+0000", 2024-09-15T01:33:01.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1800000000000001e-07 2024-09-15T01:33:01.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.256280+0000", 2024-09-15T01:33:01.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1900000000000001e-06 2024-09-15T01:33:01.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291049+0000", 2024-09-15T01:33:01.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034769125999999997 2024-09-15T01:33:01.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291054+0000", 2024-09-15T01:33:01.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7990000000000001e-06 2024-09-15T01:33:01.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291107+0000", 2024-09-15T01:33:01.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2939999999999998e-05 2024-09-15T01:33:01.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291205+0000", 2024-09-15T01:33:01.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8078999999999994e-05 2024-09-15T01:33:01.812 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.812 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.812 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.812 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:145 3.cs0 3:34a45b5f:::smithi17529310-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:01.812 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:01.812 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.467566860000002, 2024-09-15T01:33:01.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15351202899999999, 2024-09-15T01:33:01.813 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.813 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:01.813 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.813 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.813 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.813 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 145 2024-09-15T01:33:01.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.813 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:01.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:01.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:01.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0500000000000004e-07 2024-09-15T01:33:01.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297871+0000", 2024-09-15T01:33:01.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9769999999999999e-06 2024-09-15T01:33:01.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297878+0000", 2024-09-15T01:33:01.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9120000000000001e-06 2024-09-15T01:33:01.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297882+0000", 2024-09-15T01:33:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0910000000000003e-06 2024-09-15T01:33:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.298932+0000", 2024-09-15T01:33:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00104904 2024-09-15T01:33:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.299009+0000", 2024-09-15T01:33:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7095000000000006e-05 2024-09-15T01:33:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.306936+0000", 2024-09-15T01:33:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0079278730000000002 2024-09-15T01:33:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.307022+0000", 2024-09-15T01:33:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5254999999999996e-05 2024-09-15T01:33:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.451381+0000", 2024-09-15T01:33:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14435928100000001 2024-09-15T01:33:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=81, last_complete=44'30, committed=1, applied=1))", 2024-09-15T01:33:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.242579+0000", 2024-09-15T01:33:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.522856436, 2024-09-15T01:33:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033297823999999997, 2024-09-15T01:33:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242579+0000", 2024-09-15T01:33:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242579+0000", 2024-09-15T01:33:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242580+0000", 2024-09-15T01:33:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3499999999999995e-07 2024-09-15T01:33:01.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242585+0000", 2024-09-15T01:33:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8400000000000002e-06 2024-09-15T01:33:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275688+0000", 2024-09-15T01:33:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033103399999999998 2024-09-15T01:33:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275692+0000", 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9349999999999996e-06 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275708+0000", 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5930000000000002e-05 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275877+0000", 2024-09-15T01:33:01.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016898400000000001 2024-09-15T01:33:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:312 3.cs0 3:37dffec7:::smithi17529310-13:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.487243245, 2024-09-15T01:33:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770305000000002, 2024-09-15T01:33:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.823 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 312 2024-09-15T01:33:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0200000000000002e-07 2024-09-15T01:33:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278196+0000", 2024-09-15T01:33:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7319999999999998e-06 2024-09-15T01:33:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278216+0000", 2024-09-15T01:33:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0375999999999999e-05 2024-09-15T01:33:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278221+0000", 2024-09-15T01:33:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4889999999999997e-06 2024-09-15T01:33:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279668+0000", 2024-09-15T01:33:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014473929999999999 2024-09-15T01:33:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279753+0000", 2024-09-15T01:33:01.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4567999999999994e-05 2024-09-15T01:33:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280213+0000", 2024-09-15T01:33:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00046030399999999998 2024-09-15T01:33:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280327+0000", 2024-09-15T01:33:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011404399999999999 2024-09-15T01:33:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280342+0000", 2024-09-15T01:33:01.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4403000000000001e-05 2024-09-15T01:33:01.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280467+0000", 2024-09-15T01:33:01.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000125594 2024-09-15T01:33:01.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280562+0000", 2024-09-15T01:33:01.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5317999999999998e-05 2024-09-15T01:33:01.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:01.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328870+0000", 2024-09-15T01:33:01.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048307782 2024-09-15T01:33:01.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328963+0000", 2024-09-15T01:33:01.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2700000000000004e-05 2024-09-15T01:33:01.831 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.831 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.831 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.831 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:313 3.cs0 3:37dffec7:::smithi17529310-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:01.831 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:01.831 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.487210363999999, 2024-09-15T01:33:01.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053267040000000002, 2024-09-15T01:33:01.831 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.831 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:01.832 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.832 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.832 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.832 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 313 2024-09-15T01:33:01.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.832 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:01.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:01.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278226+0000", 2024-09-15T01:33:01.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7100000000000002e-07 2024-09-15T01:33:01.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278228+0000", 2024-09-15T01:33:01.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1859999999999999e-06 2024-09-15T01:33:01.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278235+0000", 2024-09-15T01:33:01.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9020000000000002e-06 2024-09-15T01:33:01.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278240+0000", 2024-09-15T01:33:01.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9540000000000003e-06 2024-09-15T01:33:01.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280404+0000", 2024-09-15T01:33:01.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021641469999999999 2024-09-15T01:33:01.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:01.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280503+0000", 2024-09-15T01:33:01.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9090000000000002e-05 2024-09-15T01:33:01.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328996+0000", 2024-09-15T01:33:01.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048492533999999997 2024-09-15T01:33:01.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.329053+0000", 2024-09-15T01:33:01.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6997999999999999e-05 2024-09-15T01:33:01.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.331493+0000", 2024-09-15T01:33:01.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024397580000000002 2024-09-15T01:33:01.837 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.837 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.837 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.837 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=95, last_complete=44'34, committed=1, applied=1))", 2024-09-15T01:33:01.837 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:01.837 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.483806020999999, 2024-09-15T01:33:01.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047344538999999998, 2024-09-15T01:33:01.838 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.838 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.838 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:01.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:01.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:01.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7700000000000001e-07 2024-09-15T01:33:01.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281632+0000", 2024-09-15T01:33:01.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.562e-06 2024-09-15T01:33:01.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328775+0000", 2024-09-15T01:33:01.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047143813 2024-09-15T01:33:01.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328779+0000", 2024-09-15T01:33:01.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.895e-06 2024-09-15T01:33:01.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328803+0000", 2024-09-15T01:33:01.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4076000000000001e-05 2024-09-15T01:33:01.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328974+0000", 2024-09-15T01:33:01.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017101599999999999 2024-09-15T01:33:01.841 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.841 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.841 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.842 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:324 3.cs0 3:34a45b5f:::smithi17529310-48:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:01.842 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:01.842 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.434790914000001, 2024-09-15T01:33:01.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18862163400000001, 2024-09-15T01:33:01.842 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.842 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:01.842 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.842 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.842 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.842 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 324 2024-09-15T01:33:01.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.843 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:01.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:01.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:01.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699999999999999e-07 2024-09-15T01:33:01.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330648+0000", 2024-09-15T01:33:01.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.694e-06 2024-09-15T01:33:01.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330654+0000", 2024-09-15T01:33:01.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4090000000000002e-06 2024-09-15T01:33:01.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330659+0000", 2024-09-15T01:33:01.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3210000000000004e-06 2024-09-15T01:33:01.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332198+0000", 2024-09-15T01:33:01.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015388699999999999 2024-09-15T01:33:01.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332262+0000", 2024-09-15T01:33:01.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4196000000000007e-05 2024-09-15T01:33:01.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332785+0000", 2024-09-15T01:33:01.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052292099999999998 2024-09-15T01:33:01.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332870+0000", 2024-09-15T01:33:01.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5049999999999994e-05 2024-09-15T01:33:01.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332877+0000", 2024-09-15T01:33:01.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7769999999999997e-06 2024-09-15T01:33:01.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332902+0000", 2024-09-15T01:33:01.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5012e-05 2024-09-15T01:33:01.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332962+0000", 2024-09-15T01:33:01.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0649999999999997e-05 2024-09-15T01:33:01.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:01.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519220+0000", 2024-09-15T01:33:01.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18625739199999999 2024-09-15T01:33:01.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519267+0000", 2024-09-15T01:33:01.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6875000000000001e-05 2024-09-15T01:33:01.849 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.849 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.849 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.850 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:325 3.cs0 3:34a45b5f:::smithi17529310-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:01.850 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:01.850 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.434772851, 2024-09-15T01:33:01.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.189890015, 2024-09-15T01:33:01.850 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.850 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:01.850 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.850 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.850 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.851 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 325 2024-09-15T01:33:01.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.851 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:01.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:01.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:01.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-15T01:33:01.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330667+0000", 2024-09-15T01:33:01.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4410000000000002e-06 2024-09-15T01:33:01.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330673+0000", 2024-09-15T01:33:01.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5429999999999999e-06 2024-09-15T01:33:01.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330676+0000", 2024-09-15T01:33:01.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.683e-06 2024-09-15T01:33:01.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333439+0000", 2024-09-15T01:33:01.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002763415 2024-09-15T01:33:01.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:01.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333492+0000", 2024-09-15T01:33:01.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2760000000000003e-05 2024-09-15T01:33:01.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519496+0000", 2024-09-15T01:33:01.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18600408199999999 2024-09-15T01:33:01.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519560+0000", 2024-09-15T01:33:01.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3660999999999999e-05 2024-09-15T01:33:01.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.520553+0000", 2024-09-15T01:33:01.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00099301399999999996 2024-09-15T01:33:01.855 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.855 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.856 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.856 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=100, last_complete=44'39, committed=1, applied=1))", 2024-09-15T01:33:01.856 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:01.856 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.432370808, 2024-09-15T01:33:01.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18640773799999999, 2024-09-15T01:33:01.856 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.856 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.856 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:01.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:01.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:01.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5999999999999999e-08 2024-09-15T01:33:01.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333066+0000", 2024-09-15T01:33:01.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.043e-06 2024-09-15T01:33:01.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519088+0000", 2024-09-15T01:33:01.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18602205399999999 2024-09-15T01:33:01.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519092+0000", 2024-09-15T01:33:01.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0890000000000002e-06 2024-09-15T01:33:01.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519171+0000", 2024-09-15T01:33:01.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8857999999999996e-05 2024-09-15T01:33:01.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519473+0000", 2024-09-15T01:33:01.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030159799999999998 2024-09-15T01:33:01.861 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.861 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.861 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.861 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 47/31 ECSubWriteReply(tid=113, last_complete=47'51, committed=1, applied=1))", 2024-09-15T01:33:01.861 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:01.861 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.505548997, 2024-09-15T01:33:01.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.63648575600000001, 2024-09-15T01:33:01.862 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.862 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.862 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:01.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:01.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:01.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-07 2024-09-15T01:33:01.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259890+0000", 2024-09-15T01:33:01.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5859999999999999e-06 2024-09-15T01:33:01.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.895924+0000", 2024-09-15T01:33:01.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.63603432699999995 2024-09-15T01:33:01.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.895929+0000", 2024-09-15T01:33:01.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1340000000000003e-06 2024-09-15T01:33:01.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.896000+0000", 2024-09-15T01:33:01.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0687999999999998e-05 2024-09-15T01:33:01.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.896373+0000", 2024-09-15T01:33:01.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037265200000000001 2024-09-15T01:33:01.866 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.866 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.866 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.866 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 48/31 ECSubWriteReply(tid=128, last_complete=48'57, committed=1, applied=1))", 2024-09-15T01:33:01.866 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:01.866 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.458733508, 2024-09-15T01:33:01.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18111660500000001, 2024-09-15T01:33:01.866 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.867 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.867 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:01.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:01.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306703+0000", 2024-09-15T01:33:01.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3e-07 2024-09-15T01:33:01.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306707+0000", 2024-09-15T01:33:01.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7500000000000001e-06 2024-09-15T01:33:01.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487741+0000", 2024-09-15T01:33:01.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18103418199999999 2024-09-15T01:33:01.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487745+0000", 2024-09-15T01:33:01.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2189999999999998e-06 2024-09-15T01:33:01.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487789+0000", 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3430000000000003e-05 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487819+0000", 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0394000000000001e-05 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.870 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 53/31 ECSubWriteReply(tid=146, last_complete=53'74, committed=1, applied=1))", 2024-09-15T01:33:01.871 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:01.871 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.5648920589999999, 2024-09-15T01:33:01.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.74248160299999999, 2024-09-15T01:33:01.871 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.871 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.871 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:01.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:01.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:01.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-07 2024-09-15T01:33:01.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200550+0000", 2024-09-15T01:33:01.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5910000000000003e-06 2024-09-15T01:33:01.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.942942+0000", 2024-09-15T01:33:01.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.74239190499999996 2024-09-15T01:33:01.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.942965+0000", 2024-09-15T01:33:01.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2983999999999999e-05 2024-09-15T01:33:01.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.943000+0000", 2024-09-15T01:33:01.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5336999999999997e-05 2024-09-15T01:33:01.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.943025+0000", 2024-09-15T01:33:01.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5250999999999999e-05 2024-09-15T01:33:01.874 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.874 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.875 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.875 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.7s6 55/52 [PushOp(3:e6e8875f:::smithi17529310-31:head, version: 50'31, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:e6e8875f:::smithi17529310-31:head@50'31, size: 1949696, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:33:01.875 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:01.875 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.6671774539999999, 2024-09-15T01:33:01.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18071695199999999, 2024-09-15T01:33:01.875 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.875 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:01.875 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:01.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098259+0000", 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0200000000000001e-07 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098545+0000", 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028574699999999999 2024-09-15T01:33:01.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278693+0000", 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.180148009 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278700+0000", 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.888e-06 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278803+0000", 2024-09-15T01:33:01.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102766 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278975+0000", 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017284000000000001 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:524 3.fs0 3:f8e751c8:::smithi17529310-17:head [setxattr _header (57) in=7b] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.671412981, 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.115556992, 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 524 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2200000000000004e-07 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094028+0000", 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9710000000000003e-06 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094040+0000", 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1066e-05 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094044+0000", 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8770000000000001e-06 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095122+0000", 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010771839999999999 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095188+0000", 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6298999999999997e-05 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095521+0000", 2024-09-15T01:33:01.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033339600000000002 2024-09-15T01:33:01.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:01.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095614+0000", 2024-09-15T01:33:01.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2295999999999999e-05 2024-09-15T01:33:01.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:01.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095620+0000", 2024-09-15T01:33:01.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8000000000000001e-06 2024-09-15T01:33:01.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:01.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095653+0000", 2024-09-15T01:33:01.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2867000000000002e-05 2024-09-15T01:33:01.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:01.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095740+0000", 2024-09-15T01:33:01.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6300999999999999e-05 2024-09-15T01:33:01.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:01.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209497+0000", 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.113757184 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209580+0000", 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3128999999999996e-05 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:525 3.fs0 3:f8e751c8:::smithi17529310-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:01.886 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.6713871559999998, 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11664384, 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 525 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:01.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6100000000000001e-07 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:01.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094051+0000", 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1950000000000002e-06 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094058+0000", 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4350000000000002e-06 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094063+0000", 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2039999999999996e-06 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095664+0000", 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001600642 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095725+0000", 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1707999999999999e-05 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209923+0000", 2024-09-15T01:33:01.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.114197744 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209997+0000", 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3341000000000001e-05 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.210693+0000", 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00069601000000000005 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.891 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:01.892 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:01.892 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:01.893 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-15T01:33:01.894 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:01.891+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:01.992 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:01.992 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:01.992 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:01.992 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:01.994 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:01.991+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:02.000 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-15T01:33:02.086 INFO:tasks.rados.rados.0.smithi175.stdout:209: done (3 left) 2024-09-15T01:33:02.086 INFO:tasks.rados.rados.0.smithi175.stdout:210: done (2 left) 2024-09-15T01:33:02.086 INFO:tasks.rados.rados.0.smithi175.stdout:211: done (1 left) 2024-09-15T01:33:02.086 INFO:tasks.rados.rados.0.smithi175.stdout:212: done (0 left) 2024-09-15T01:33:02.086 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 2 current snap is 14 2024-09-15T01:33:02.086 INFO:tasks.rados.rados.0.smithi175.stdout:213: seq_num 110 ranges {32768=688128,720896=507904,1228800=655360} 2024-09-15T01:33:02.090 INFO:tasks.rados.rados.0.smithi175.stdout:213: writing smithi17529310-2 from 32768 to 720896 tid 1 2024-09-15T01:33:02.093 INFO:tasks.rados.rados.0.smithi175.stdout:213: writing smithi17529310-2 from 720896 to 1228800 tid 2 2024-09-15T01:33:02.094 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:02.092+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:02.097 INFO:tasks.rados.rados.0.smithi175.stdout:213: writing smithi17529310-2 from 1228800 to 1884160 tid 3 2024-09-15T01:33:02.097 INFO:tasks.rados.rados.0.smithi175.stdout:214: delete oid 6 current snap is 14 2024-09-15T01:33:02.099 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:02.099 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:02.100 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:02.100 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:02.100 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:02.100 INFO:tasks.rados.rados.0.smithi175.stdout:214: done (1 left) 2024-09-15T01:33:02.100 INFO:tasks.rados.rados.0.smithi175.stdout:215: copy_from oid 14 from oid 7 current snap is 14 2024-09-15T01:33:02.100 INFO:tasks.rados.rados.0.smithi175.stdout:216: snap_create 2024-09-15T01:33:02.104 INFO:tasks.rados.rados.0.smithi175.stdout:215: finishing copy_from racing read to smithi17529310-14 2024-09-15T01:33:02.104 INFO:tasks.rados.rados.0.smithi175.stdout:215: finishing copy_from to smithi17529310-14 2024-09-15T01:33:02.104 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 14 v 14 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-15T01:33:02.108 INFO:tasks.rados.rados.0.smithi175.stdout:213: finishing write tid 1 to smithi17529310-2 2024-09-15T01:33:02.108 INFO:tasks.rados.rados.0.smithi175.stdout:213: finishing write tid 2 to smithi17529310-2 2024-09-15T01:33:02.108 INFO:tasks.rados.rados.0.smithi175.stdout:213: finishing write tid 3 to smithi17529310-2 2024-09-15T01:33:02.108 INFO:tasks.rados.rados.0.smithi175.stdout:213: finishing write tid 4 to smithi17529310-2 2024-09-15T01:33:02.110 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-15T01:33:02.110 INFO:tasks.rados.rados.0.smithi175.stdout:213: finishing write tid 5 to smithi17529310-2 2024-09-15T01:33:02.110 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 2 v 98 (ObjNum 110 snap 14 seq_num 110) dirty exists 2024-09-15T01:33:02.111 INFO:tasks.rados.rados.0.smithi175.stdout:213: left oid 2 (ObjNum 110 snap 14 seq_num 110) 2024-09-15T01:33:02.195 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:02.192+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:02.208 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:02.208 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:02.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:02.208 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:02.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.208 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:10 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [append 0~65536,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:02.208 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.175773+0000", 2024-09-15T01:33:02.208 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.029336037, 2024-09-15T01:33:02.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011565234000000001, 2024-09-15T01:33:02.208 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.208 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 10 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175773+0000", 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175773+0000", 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175775+0000", 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.035e-06 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175820+0000", 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5272999999999998e-05 2024-09-15T01:33:02.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175836+0000", 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6350000000000001e-05 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175844+0000", 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8620000000000003e-06 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.175890+0000", 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.617e-05 2024-09-15T01:33:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176032+0000", 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141426 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176738+0000", 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00070667099999999997 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176861+0000", 2024-09-15T01:33:02.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012252899999999999 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176905+0000", 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3761000000000003e-05 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176965+0000", 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9910000000000001e-05 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176985+0000", 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0047999999999998e-05 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187276+0000", 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010291664000000001 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.187338+0000", 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1535e-05 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.214 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.215 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:11 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:02.215 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:02.215 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.028119295, 2024-09-15T01:33:02.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032033109999999997, 2024-09-15T01:33:02.215 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.215 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:02.215 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:02.215 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:02.215 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.215 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 11 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176990+0000", 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2200000000000002e-07 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176995+0000", 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8040000000000001e-06 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177006+0000", 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1276e-05 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177012+0000", 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9370000000000001e-06 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177055+0000", 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2472999999999998e-05 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177197+0000", 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142519 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177567+0000", 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036945000000000001 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177659+0000", 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1928999999999993e-05 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177694+0000", 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4885000000000002e-05 2024-09-15T01:33:02.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177722+0000", 2024-09-15T01:33:02.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8334e-05 2024-09-15T01:33:02.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177733+0000", 2024-09-15T01:33:02.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1491000000000001e-05 2024-09-15T01:33:02.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208963+0000", 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031230106000000001 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209022+0000", 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8984000000000003e-05 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:12 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.027369026999999, 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033440417, 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 12 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:02.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2500000000000005e-07 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177745+0000", 2024-09-15T01:33:02.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7269999999999998e-06 2024-09-15T01:33:02.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177755+0000", 2024-09-15T01:33:02.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9000000000000001e-06 2024-09-15T01:33:02.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177760+0000", 2024-09-15T01:33:02.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1150000000000002e-06 2024-09-15T01:33:02.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177801+0000", 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0802e-05 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177877+0000", 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6604e-05 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209067+0000", 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031189505999999999 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209144+0000", 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6728000000000001e-05 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211180+0000", 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020363099999999999 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.227 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=12, reqid=client.4477.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:02.227 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:02.227 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.015164813999998, 2024-09-15T01:33:02.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022201101000000001, 2024-09-15T01:33:02.227 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.227 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.227 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:02.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:02.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:02.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.75e-07 2024-09-15T01:33:02.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189957+0000", 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2531000000000001e-05 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211941+0000", 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021983546999999999 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211947+0000", 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7329999999999997e-06 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211977+0000", 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9607e-05 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211982+0000", 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6290000000000001e-06 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212053+0000", 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1890000000000005e-05 2024-09-15T01:33:02.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212060+0000", 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3339999999999998e-06 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212087+0000", 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7314e-05 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212098+0000", 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1141999999999999e-05 2024-09-15T01:33:02.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212145+0000", 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699000000000001e-05 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:72 3.5s0 3:af8bec4d:::smithi17529310-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.99779208, 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.037268951000000002, 2024-09-15T01:33:02.232 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.34e-07 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207319+0000", 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207326+0000", 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4860000000000002e-06 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207330+0000", 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0690000000000003e-06 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208687+0000", 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001356966 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208760+0000", 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2663000000000005e-05 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210380+0000", 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001620659 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210464+0000", 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3387999999999995e-05 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.244586+0000", 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034121925999999997 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.0s2 34/31 ECSubWrite(tid=15, reqid=client.4477.0:74, at_version=34'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.992681333, 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069361051000000007, 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.237 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212428+0000", 2024-09-15T01:33:02.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3399999999999999e-07 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212444+0000", 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5681000000000001e-05 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281357+0000", 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068913881999999996 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281366+0000", 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9740000000000008e-06 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281415+0000", 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8498000000000002e-05 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281421+0000", 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4590000000000003e-06 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281487+0000", 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5590000000000001e-05 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281500+0000", 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3298e-05 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281575+0000", 2024-09-15T01:33:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5247000000000006e-05 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281597+0000", 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1872999999999999e-05 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281788+0000", 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000191015 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=20, reqid=client.4477.0:131, at_version=0'0, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.282692+0000", 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.922417073999998, 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01281827, 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.282692+0000", 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.282692+0000", 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.282692+0000", 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.68e-07 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.282702+0000", 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8230000000000006e-06 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295220+0000", 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012517916 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295229+0000", 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.867e-06 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295274+0000", 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5583e-05 2024-09-15T01:33:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295283+0000", 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1880000000000002e-06 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295360+0000", 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7578999999999997e-05 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295372+0000", 2024-09-15T01:33:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.207e-05 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295419+0000", 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7289000000000001e-05 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295443+0000", 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3714999999999999e-05 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.295510+0000", 2024-09-15T01:33:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6672000000000001e-05 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 38/31 ECSubWrite(tid=17, reqid=client.4477.0:208, at_version=38'12, trim_to=0'0, roll_forward_to=36'8))", 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.983747435000002, 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.93429882900000005, 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221362+0000", 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9699999999999996e-07 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221380+0000", 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.804e-05 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.154929+0000", 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.93354887600000003 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.154938+0000", 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6220000000000004e-06 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155018+0000", 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0136000000000007e-05 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155027+0000", 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8259999999999999e-06 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155116+0000", 2024-09-15T01:33:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8939999999999999e-05 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155127+0000", 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0981e-05 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155185+0000", 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7828999999999998e-05 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155204+0000", 2024-09-15T01:33:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8649999999999999e-05 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155660+0000", 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000456432 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.270979+0000", 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.934129528, 2024-09-15T01:33:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010394718000000001, 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.270979+0000", 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.270979+0000", 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.270980+0000", 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.61e-07 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271003+0000", 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2986e-05 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271041+0000", 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8056000000000003e-05 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271048+0000", 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0790000000000003e-06 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271096+0000", 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7956e-05 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271105+0000", 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6680000000000007e-06 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271191+0000", 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6670999999999997e-05 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271205+0000", 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3667e-05 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271245+0000", 2024-09-15T01:33:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0099999999999999e-05 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271259+0000", 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4282e-05 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281374+0000", 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010114491999999999 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s3 44/38 ECSubWrite(tid=22, reqid=client.4477.0:293, at_version=44'14, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.271270+0000", 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.933838973, 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015434979, 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271270+0000", 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271270+0000", 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271270+0000", 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1999999999999999e-07 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271482+0000", 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021116799999999999 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271510+0000", 2024-09-15T01:33:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8881999999999999e-05 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271517+0000", 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3609999999999997e-06 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271571+0000", 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4193000000000001e-05 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271577+0000", 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3849999999999999e-06 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271936+0000", 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035823400000000003 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271967+0000", 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1644999999999999e-05 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.272044+0000", 2024-09-15T01:33:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6448999999999999e-05 2024-09-15T01:33:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.272061+0000", 2024-09-15T01:33:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7524999999999999e-05 2024-09-15T01:33:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.286705+0000", 2024-09-15T01:33:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014643417000000001 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 44/31 ECSubWrite(tid=34, reqid=client.4477.0:296, at_version=44'25, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.271499+0000", 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.933609148, 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010309265, 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271499+0000", 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271499+0000", 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271500+0000", 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5199999999999995e-07 2024-09-15T01:33:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271532+0000", 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1259e-05 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271567+0000", 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5729000000000003e-05 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271576+0000", 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9509999999999995e-06 2024-09-15T01:33:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271616+0000", 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9601000000000003e-05 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271624+0000", 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3590000000000008e-06 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271712+0000", 2024-09-15T01:33:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8096000000000005e-05 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271737+0000", 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5040000000000001e-05 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271776+0000", 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8594999999999999e-05 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271793+0000", 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7428e-05 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281809+0000", 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010015355 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:314 3.5s0 3:af8bec4d:::smithi17529310-24:head [delete] snapc 8=[8] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278138+0000", 2024-09-15T01:33:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.926970365000001, 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01242359, 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 314 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278138+0000", 2024-09-15T01:33:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278138+0000", 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278140+0000", 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0650000000000001e-06 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278145+0000", 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4870000000000004e-06 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278162+0000", 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6779999999999999e-05 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278170+0000", 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278228+0000", 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8638000000000003e-05 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278329+0000", 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010108 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279378+0000", 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010488139999999999 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279703+0000", 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000324333 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279716+0000", 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3417999999999999e-05 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279760+0000", 2024-09-15T01:33:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4388999999999998e-05 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279820+0000", 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9320000000000001e-05 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281759+0000", 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001939412 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.290562+0000", 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00880272 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 51/31 ECSubWrite(tid=56, reqid=osd.5.0:56, at_version=51'43, trim_to=0'0, roll_forward_to=50'41))", 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1282577010000008, 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.063539848999999995, 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076876+0000", 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4834999999999999e-05 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139053+0000", 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.062177018000000001 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139067+0000", 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3842999999999999e-05 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139206+0000", 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013896500000000001 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139221+0000", 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5193e-05 2024-09-15T01:33:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139472+0000", 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025116 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139504+0000", 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1282999999999998e-05 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139541+0000", 2024-09-15T01:33:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7518999999999998e-05 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139557+0000", 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5781e-05 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.140391+0000", 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00083395699999999997 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:505 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [copy-from ver 2] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.132070358, 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15814687399999999, 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 505 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073040+0000", 2024-09-15T01:33:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.792e-06 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073047+0000", 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0169999999999999e-06 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073064+0000", 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.668e-05 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073081+0000", 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7000999999999999e-05 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073158+0000", 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6779000000000005e-05 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073284+0000", 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000126802 2024-09-15T01:33:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082517+0000", 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0092324069999999998 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082822+0000", 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030544099999999999 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082921+0000", 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8760999999999999e-05 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082994+0000", 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2769999999999996e-05 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083217+0000", 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000223706 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231110+0000", 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.147892096 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231185+0000", 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5622000000000002e-05 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 55/31 ECSubWriteReply(tid=49, last_complete=55'33, committed=1, applied=1))", 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.1208270450000004, 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14718672399999999, 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9599999999999998e-07 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084287+0000", 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.230928+0000", 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14664121799999999 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.230933+0000", 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4089999999999996e-06 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231018+0000", 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5589999999999999e-05 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231468+0000", 2024-09-15T01:33:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045001999999999998 2024-09-15T01:33:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s4 56/54 ECSubWrite(tid=228, reqid=client.4477.0:520, at_version=56'33, trim_to=0'0, roll_forward_to=55'31))", 2024-09-15T01:33:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.1311070140000004, 2024-09-15T01:33:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016830581000000001, 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7000000000000003e-07 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074027+0000", 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4845e-05 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074072+0000", 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4583000000000003e-05 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074080+0000", 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6699999999999994e-06 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074280+0000", 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020015699999999999 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074290+0000", 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0586e-05 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074562+0000", 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027202599999999997 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074573+0000", 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0974e-05 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074624+0000", 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0385e-05 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074786+0000", 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016278399999999999 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.090832+0000", 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016045700999999999 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 56/31 ECSubWrite(tid=115, reqid=osd.14.0:115, at_version=56'45, trim_to=0'0, roll_forward_to=54'41))", 2024-09-15T01:33:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.079340+0000", 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.1257685469999998, 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011653982, 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079340+0000", 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079340+0000", 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079340+0000", 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7300000000000002e-07 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079362+0000", 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1401000000000001e-05 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079434+0000", 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2611e-05 2024-09-15T01:33:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079444+0000", 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4560000000000006e-06 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079502+0000", 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8201000000000001e-05 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079511+0000", 2024-09-15T01:33:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2769999999999994e-06 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079765+0000", 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025312000000000002 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079836+0000", 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1239e-05 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079886+0000", 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0436000000000003e-05 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079959+0000", 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2631000000000003e-05 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.090994+0000", 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011035237 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 56/31 ECSubWrite(tid=70, reqid=osd.5.0:69, at_version=0'0, trim_to=0'0, roll_forward_to=56'55))", 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.079609+0000", 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.1254994710000004, 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011689662999999999, 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079609+0000", 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079609+0000", 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079610+0000", 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4399999999999999e-07 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079623+0000", 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2639e-05 2024-09-15T01:33:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079659+0000", 2024-09-15T01:33:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6239999999999999e-05 2024-09-15T01:33:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079665+0000", 2024-09-15T01:33:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.516e-06 2024-09-15T01:33:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079712+0000", 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6746000000000003e-05 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079722+0000", 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6760000000000002e-06 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080756+0000", 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010342389999999999 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080767+0000", 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0869e-05 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082434+0000", 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0016665930000000001 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082463+0000", 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9212000000000001e-05 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.091299+0000", 2024-09-15T01:33:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0088360890000000001 2024-09-15T01:33:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s4 56/54 ECSubWrite(tid=229, reqid=client.4477.0:520, at_version=0'0, trim_to=0'0, roll_forward_to=56'33))", 2024-09-15T01:33:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.079988+0000", 2024-09-15T01:33:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.1251206649999999, 2024-09-15T01:33:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01122911, 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079988+0000", 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079988+0000", 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079989+0000", 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.92e-07 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080012+0000", 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3212000000000001e-05 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080065+0000", 2024-09-15T01:33:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2735999999999998e-05 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080072+0000", 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4889999999999998e-06 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080131+0000", 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.906e-05 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080142+0000", 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0488e-05 2024-09-15T01:33:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080251+0000", 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010946 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080294+0000", 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3331999999999998e-05 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080348+0000", 2024-09-15T01:33:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3634999999999997e-05 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080366+0000", 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8425000000000001e-05 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.091217+0000", 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010850581 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s3 56/38 ECSubWrite(tid=48, reqid=osd.0.0:48, at_version=56'29, trim_to=0'0, roll_forward_to=54'27))", 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.080428+0000", 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.1246809520000003, 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01186971, 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080428+0000", 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080428+0000", 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080428+0000", 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6499999999999998e-07 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080455+0000", 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6208999999999999e-05 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080522+0000", 2024-09-15T01:33:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7298000000000005e-05 2024-09-15T01:33:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080528+0000", 2024-09-15T01:33:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4010000000000002e-06 2024-09-15T01:33:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080614+0000", 2024-09-15T01:33:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5830999999999999e-05 2024-09-15T01:33:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.080624+0000", 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8740000000000007e-06 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082696+0000", 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020717589999999998 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082733+0000", 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7533000000000001e-05 2024-09-15T01:33:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082770+0000", 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6216e-05 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082782+0000", 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2415999999999999e-05 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.092297+0000", 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0095154079999999995 2024-09-15T01:33:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:02.319 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:33:02.322 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:33:02.322 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:33:02.324 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-15T01:33:02.082276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":71811635,"num_objects":58,"num_object_clones":12,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":3,"num_read":337,"num_read_kb":41893,"num_write":585,"num_write_kb":91346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":20021248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":15,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1405194240,"kb_used":12162128,"kb_used_data":1393053764,"kb_used_omap":15747,"kb_used_meta":0,"kb_avail":1393032112,"statfs":{"total":1438918901760,"available":1426464882688,"internally_reserved":22171648,"allocated":1426487054336,"data_stored":1426487054336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16125116,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":25,"commit_latency_ns":0,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16351232,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":37,"num_read_kb":13087,"num_write":79,"num_write_kb":21072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":10141696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013996"},"pg_stats":[{"pgid":"3.f","version":"56'93","reported_seq":174,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T01:32:58.210671+0000","last_change":"2024-09-15T01:32:58.081834+0000","last_active":"2024-09-15T01:32:58.210671+0000","last_peered":"2024-09-15T01:32:58.210671+0000","last_clean":"2024-09-15T01:32:58.210671+0000","last_became_active":"2024-09-15T01:32:34.487273+0000","last_became_peered":"2024-09-15T01:32:34.487273+0000","last_unstale":"2024-09-15T01:32:58.210671+0000","last_undegraded":"2024-09-15T01:32:58.210671+0000","last_fullsized":"2024-09-15T01:32:58.210671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:42.230488+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0098134580000000006,"stat_sum":{"num_bytes":9715712,"num_objects":7,"num_object_clones":1,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":151,"num_write":82,"num_write_kb":12912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":113,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.086967+0000","last_change":"2024-09-15T01:33:00.087196+0000","last_active":"2024-09-15T01:33:00.086967+0000","last_peered":"2024-09-15T01:33:00.086967+0000","last_clean":"2024-09-15T01:33:00.086967+0000","last_became_active":"2024-09-15T01:32:54.091710+0000","last_became_peered":"2024-09-15T01:32:54.091710+0000","last_unstale":"2024-09-15T01:33:00.086967+0000","last_undegraded":"2024-09-15T01:33:00.086967+0000","last_fullsized":"2024-09-15T01:33:00.086967+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:55.820742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034944000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,14,1,11,8,6,12],"acting":[2,14,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"3.d","version":"58'50","reported_seq":137,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.088995+0000","last_change":"2024-09-15T01:33:00.088995+0000","last_active":"2024-09-15T01:33:00.088995+0000","last_peered":"2024-09-15T01:33:00.088995+0000","last_clean":"2024-09-15T01:33:00.088995+0000","last_became_active":"2024-09-15T01:32:34.487040+0000","last_became_peered":"2024-09-15T01:32:34.487040+0000","last_unstale":"2024-09-15T01:33:00.088995+0000","last_undegraded":"2024-09-15T01:33:00.088995+0000","last_fullsized":"2024-09-15T01:33:00.088995+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.966217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527575,"stat_sum":{"num_bytes":2473984,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":871,"num_write":41,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"3.c","version":"56'56","reported_seq":139,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T01:32:58.081582+0000","last_change":"2024-09-15T01:32:58.081582+0000","last_active":"2024-09-15T01:32:58.081582+0000","last_peered":"2024-09-15T01:32:58.081582+0000","last_clean":"2024-09-15T01:32:58.081582+0000","last_became_active":"2024-09-15T01:32:34.487072+0000","last_became_peered":"2024-09-15T01:32:34.487072+0000","last_unstale":"2024-09-15T01:32:58.081582+0000","last_undegraded":"2024-09-15T01:32:58.081582+0000","last_fullsized":"2024-09-15T01:32:58.081582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.186197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010697115,"stat_sum":{"num_bytes":7864320,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":33,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"}]},{"pgid":"3.b","version":"53'23","reported_seq":144,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.087513+0000","last_change":"2024-09-15T01:33:00.087840+0000","last_active":"2024-09-15T01:33:00.087513+0000","last_peered":"2024-09-15T01:33:00.087513+0000","last_clean":"2024-09-15T01:33:00.087513+0000","last_became_active":"2024-09-15T01:32:41.206211+0000","last_became_peered":"2024-09-15T01:32:41.206211+0000","last_unstale":"2024-09-15T01:33:00.087513+0000","last_undegraded":"2024-09-15T01:33:00.087513+0000","last_fullsized":"2024-09-15T01:33:00.087513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.489551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079285100000000002,"stat_sum":{"num_bytes":2457600,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":10,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,13],"acting":[4,0,1,12,6,7,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"3.a","version":"58'23","reported_seq":180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.091831+0000","last_change":"2024-09-15T01:33:00.093987+0000","last_active":"2024-09-15T01:33:00.091831+0000","last_peered":"2024-09-15T01:33:00.091831+0000","last_clean":"2024-09-15T01:33:00.091831+0000","last_became_active":"2024-09-15T01:32:57.079821+0000","last_became_peered":"2024-09-15T01:32:57.079821+0000","last_unstale":"2024-09-15T01:33:00.091831+0000","last_undegraded":"2024-09-15T01:33:00.091831+0000","last_fullsized":"2024-09-15T01:33:00.091831+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.988837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006026488,"stat_sum":{"num_bytes":1982464,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":10,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2359296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8,14,6,11],"acting":[3,7,1,8,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"3.9","version":"58'51","reported_seq":145,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.094046+0000","last_change":"2024-09-15T01:33:00.093392+0000","last_active":"2024-09-15T01:33:00.094046+0000","last_peered":"2024-09-15T01:33:00.094046+0000","last_clean":"2024-09-15T01:33:00.094046+0000","last_became_active":"2024-09-15T01:32:34.487117+0000","last_became_peered":"2024-09-15T01:32:34.487117+0000","last_unstale":"2024-09-15T01:33:00.094046+0000","last_undegraded":"2024-09-15T01:33:00.094046+0000","last_fullsized":"2024-09-15T01:33:00.094046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.760814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055311620000000001,"stat_sum":{"num_bytes":5046272,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":21,"num_read_kb":2602,"num_write":27,"num_write_kb":6416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"3.8","version":"56'55","reported_seq":146,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.101108+0000","last_change":"2024-09-15T01:33:00.085214+0000","last_active":"2024-09-15T01:33:00.101108+0000","last_peered":"2024-09-15T01:33:00.101108+0000","last_clean":"2024-09-15T01:33:00.101108+0000","last_became_active":"2024-09-15T01:32:34.190371+0000","last_became_peered":"2024-09-15T01:32:34.190371+0000","last_unstale":"2024-09-15T01:33:00.101108+0000","last_undegraded":"2024-09-15T01:33:00.101108+0000","last_fullsized":"2024-09-15T01:33:00.101108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:51.142885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068947699999999997,"stat_sum":{"num_bytes":11468800,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":5433,"num_write":50,"num_write_kb":11200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"1.0","version":"24'112","reported_seq":178,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.075322+0000","last_change":"2024-09-15T01:32:16.749959+0000","last_active":"2024-09-15T01:32:59.075322+0000","last_peered":"2024-09-15T01:32:59.075322+0000","last_clean":"2024-09-15T01:32:59.075322+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:32:59.075322+0000","last_undegraded":"2024-09-15T01:32:59.075322+0000","last_fullsized":"2024-09-15T01:32:59.075322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_clean_scrub_stamp":"2024-09-15T01:32:15.451214+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":45,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.075806+0000","last_change":"2024-09-15T01:32:25.147954+0000","last_active":"2024-09-15T01:32:59.075806+0000","last_peered":"2024-09-15T01:32:59.075806+0000","last_clean":"2024-09-15T01:32:59.075806+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:32:59.075806+0000","last_undegraded":"2024-09-15T01:32:59.075806+0000","last_fullsized":"2024-09-15T01:32:59.075806+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.228776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"56'33","reported_seq":163,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.084907+0000","last_change":"2024-09-15T01:33:00.085227+0000","last_active":"2024-09-15T01:33:00.084907+0000","last_peered":"2024-09-15T01:33:00.084907+0000","last_clean":"2024-09-15T01:33:00.084907+0000","last_became_active":"2024-09-15T01:32:57.985433+0000","last_became_peered":"2024-09-15T01:32:57.985433+0000","last_unstale":"2024-09-15T01:33:00.084907+0000","last_undegraded":"2024-09-15T01:33:00.084907+0000","last_fullsized":"2024-09-15T01:33:00.084907+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:55.388361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045281199999999997,"stat_sum":{"num_bytes":3719168,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":16,"num_read_kb":3846,"num_write":19,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":5652480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.078973+0000","last_change":"2024-09-15T01:32:25.143709+0000","last_active":"2024-09-15T01:32:59.078973+0000","last_peered":"2024-09-15T01:32:59.078973+0000","last_clean":"2024-09-15T01:32:59.078973+0000","last_became_active":"2024-09-15T01:32:23.134382+0000","last_became_peered":"2024-09-15T01:32:23.134382+0000","last_unstale":"2024-09-15T01:32:59.078973+0000","last_undegraded":"2024-09-15T01:32:59.078973+0000","last_fullsized":"2024-09-15T01:32:59.078973+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.326224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"53'8","reported_seq":170,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.087043+0000","last_change":"2024-09-15T01:33:00.087413+0000","last_active":"2024-09-15T01:33:00.087043+0000","last_peered":"2024-09-15T01:33:00.087043+0000","last_clean":"2024-09-15T01:33:00.087043+0000","last_became_active":"2024-09-15T01:32:58.074255+0000","last_became_peered":"2024-09-15T01:32:58.074255+0000","last_unstale":"2024-09-15T01:33:00.087043+0000","last_undegraded":"2024-09-15T01:33:00.087043+0000","last_fullsized":"2024-09-15T01:33:00.087043+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:51.030793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053781800000000002,"stat_sum":{"num_bytes":1867776,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1059,"num_write":8,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2129920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,15,5,10,2,13],"acting":[4,11,15,5,10,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.075326+0000","last_change":"2024-09-15T01:32:25.583876+0000","last_active":"2024-09-15T01:32:59.075326+0000","last_peered":"2024-09-15T01:32:59.075326+0000","last_clean":"2024-09-15T01:32:59.075326+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:32:59.075326+0000","last_undegraded":"2024-09-15T01:32:59.075326+0000","last_fullsized":"2024-09-15T01:32:59.075326+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:49.092006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"56'39","reported_seq":135,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.099298+0000","last_change":"2024-09-15T01:33:00.088424+0000","last_active":"2024-09-15T01:33:00.099298+0000","last_peered":"2024-09-15T01:33:00.099298+0000","last_clean":"2024-09-15T01:33:00.099298+0000","last_became_active":"2024-09-15T01:32:34.190384+0000","last_became_peered":"2024-09-15T01:32:34.190384+0000","last_unstale":"2024-09-15T01:33:00.099298+0000","last_undegraded":"2024-09-15T01:33:00.099298+0000","last_fullsized":"2024-09-15T01:33:00.099298+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.588055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027951300000000001,"stat_sum":{"num_bytes":819200,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3144,"num_write":16,"num_write_kb":2832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":47,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.079613+0000","last_change":"2024-09-15T01:32:25.144113+0000","last_active":"2024-09-15T01:32:59.079613+0000","last_peered":"2024-09-15T01:32:59.079613+0000","last_clean":"2024-09-15T01:32:59.079613+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:32:59.079613+0000","last_undegraded":"2024-09-15T01:32:59.079613+0000","last_fullsized":"2024-09-15T01:32:59.079613+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"58'49","reported_seq":143,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.088470+0000","last_change":"2024-09-15T01:33:00.089062+0000","last_active":"2024-09-15T01:33:00.088470+0000","last_peered":"2024-09-15T01:33:00.088470+0000","last_clean":"2024-09-15T01:33:00.088470+0000","last_became_active":"2024-09-15T01:32:34.170218+0000","last_became_peered":"2024-09-15T01:32:34.170218+0000","last_unstale":"2024-09-15T01:33:00.088470+0000","last_undegraded":"2024-09-15T01:33:00.088470+0000","last_fullsized":"2024-09-15T01:33:00.088470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.093067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040047889999999999,"stat_sum":{"num_bytes":4571136,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":21,"num_read_kb":1545,"num_write":45,"num_write_kb":5264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"3.5","version":"56'37","reported_seq":125,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T01:32:58.080172+0000","last_change":"2024-09-15T01:32:58.080172+0000","last_active":"2024-09-15T01:32:58.080172+0000","last_peered":"2024-09-15T01:32:58.080172+0000","last_clean":"2024-09-15T01:32:58.080172+0000","last_became_active":"2024-09-15T01:32:34.189813+0000","last_became_peered":"2024-09-15T01:32:34.189813+0000","last_unstale":"2024-09-15T01:32:58.080172+0000","last_undegraded":"2024-09-15T01:32:58.080172+0000","last_fullsized":"2024-09-15T01:32:58.080172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:47.814751+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0088917800000000002,"stat_sum":{"num_bytes":5455872,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":6008,"num_write":20,"num_write_kb":5712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.076663+0000","last_change":"2024-09-15T01:32:25.568260+0000","last_active":"2024-09-15T01:32:59.076663+0000","last_peered":"2024-09-15T01:32:59.076663+0000","last_clean":"2024-09-15T01:32:59.076663+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:32:59.076663+0000","last_undegraded":"2024-09-15T01:32:59.076663+0000","last_fullsized":"2024-09-15T01:32:59.076663+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:32.165806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"45'31","reported_seq":120,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.090790+0000","last_change":"2024-09-15T01:33:00.091439+0000","last_active":"2024-09-15T01:33:00.090790+0000","last_peered":"2024-09-15T01:33:00.090790+0000","last_clean":"2024-09-15T01:33:00.090790+0000","last_became_active":"2024-09-15T01:32:34.189644+0000","last_became_peered":"2024-09-15T01:32:34.189644+0000","last_unstale":"2024-09-15T01:33:00.090790+0000","last_undegraded":"2024-09-15T01:33:00.090790+0000","last_fullsized":"2024-09-15T01:33:00.090790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:02.903663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012088229999999999,"stat_sum":{"num_bytes":2359296,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":4567,"num_write":17,"num_write_kb":4528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.075219+0000","last_change":"2024-09-15T01:32:25.407396+0000","last_active":"2024-09-15T01:32:59.075219+0000","last_peered":"2024-09-15T01:32:59.075219+0000","last_clean":"2024-09-15T01:32:59.075219+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:32:59.075219+0000","last_undegraded":"2024-09-15T01:32:59.075219+0000","last_fullsized":"2024-09-15T01:32:59.075219+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:39.303667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"57'51","reported_seq":228,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.093428+0000","last_change":"2024-09-15T01:33:00.093787+0000","last_active":"2024-09-15T01:33:00.093428+0000","last_peered":"2024-09-15T01:33:00.093428+0000","last_clean":"2024-09-15T01:33:00.093428+0000","last_became_active":"2024-09-15T01:32:59.541051+0000","last_became_peered":"2024-09-15T01:32:59.541051+0000","last_unstale":"2024-09-15T01:33:00.093428+0000","last_undegraded":"2024-09-15T01:33:00.093428+0000","last_fullsized":"2024-09-15T01:33:00.093428+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:38.634992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006516651,"stat_sum":{"num_bytes":9322496,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":5768,"num_write":26,"num_write_kb":9264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":8208384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,0,8,5,11],"acting":[4,6,1,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:32:59.078898+0000","last_change":"2024-09-15T01:32:25.568523+0000","last_active":"2024-09-15T01:32:59.078898+0000","last_peered":"2024-09-15T01:32:59.078898+0000","last_clean":"2024-09-15T01:32:59.078898+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:32:59.078898+0000","last_undegraded":"2024-09-15T01:32:59.078898+0000","last_fullsized":"2024-09-15T01:32:59.078898+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"56'29","reported_seq":161,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:33:00.090690+0000","last_change":"2024-09-15T01:33:00.091140+0000","last_active":"2024-09-15T01:33:00.090690+0000","last_peered":"2024-09-15T01:33:00.090690+0000","last_clean":"2024-09-15T01:33:00.090690+0000","last_became_active":"2024-09-15T01:32:41.203815+0000","last_became_peered":"2024-09-15T01:32:41.203815+0000","last_unstale":"2024-09-15T01:33:00.090690+0000","last_undegraded":"2024-09-15T01:33:00.090690+0000","last_fullsized":"2024-09-15T01:33:00.090690+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:48.344390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060834599999999995,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":6811,"num_write":34,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,10],"acting":[0,13,1,15,12,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"4"},{"start":"14","length":"1"},{"start":"17","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T01:32:58.071334+0000","last_change":"2024-09-15T01:32:25.582726+0000","last_active":"2024-09-15T01:32:58.071334+0000","last_peered":"2024-09-15T01:32:58.071334+0000","last_clean":"2024-09-15T01:32:58.071334+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:32:58.071334+0000","last_undegraded":"2024-09-15T01:32:58.071334+0000","last_fullsized":"2024-09-15T01:32:58.071334+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.907962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":71221248,"num_objects":54,"num_object_clones":12,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":3,"num_read":251,"num_read_kb":41820,"num_write":438,"num_write_kb":88672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":20021248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":628,"ondisk_log_size":628,"up":112,"acting":112,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313229,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810240,"kb_used_data":92871476,"kb_used_omap":1198,"kb_used_meta":0,"kb_avail":92869376,"statfs":{"total":95927926784,"available":95098241024,"internally_reserved":2150400,"allocated":95100391424,"data_stored":95100391424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1227693,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":3,"commit_latency_ns":0,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":574.26900000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":9,"up_from":0,"seq":90194313227,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":21,"seq":90194313228,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":803396,"kb_used_data":92876900,"kb_used_omap":639,"kb_used_meta":0,"kb_avail":92876220,"statfs":{"total":95927926784,"available":95105249280,"internally_reserved":696320,"allocated":95105945600,"data_stored":95105945600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":654916,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,8,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":3,"commit_latency_ns":0,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":0,"up_from":21,"seq":90194313229,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":820464,"kb_used_data":92861244,"kb_used_omap":1584,"kb_used_meta":0,"kb_avail":92859152,"statfs":{"total":95927926784,"available":95087771648,"internally_reserved":2142208,"allocated":95089913856,"data_stored":95089913856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1622327,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":13,"up_from":21,"seq":90194313228,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810652,"kb_used_data":92870592,"kb_used_omap":1350,"kb_used_meta":0,"kb_avail":92868964,"statfs":{"total":95927926784,"available":95097819136,"internally_reserved":1667072,"allocated":95099486208,"data_stored":95099486208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1383364,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":2,"up_from":21,"seq":90194313227,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":813364,"kb_used_data":92868836,"kb_used_omap":1210,"kb_used_meta":0,"kb_avail":92866252,"statfs":{"total":95927926784,"available":95095042048,"internally_reserved":2646016,"allocated":95097688064,"data_stored":95097688064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1239091,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":15,"up_from":21,"seq":90194313228,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810368,"kb_used_data":92870992,"kb_used_omap":1241,"kb_used_meta":0,"kb_avail":92869248,"statfs":{"total":95927926784,"available":95098109952,"internally_reserved":1785856,"allocated":95099895808,"data_stored":95099895808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1271807,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":4,"up_from":21,"seq":90194313228,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":807872,"kb_used_data":92872156,"kb_used_omap":844,"kb_used_meta":0,"kb_avail":92871744,"statfs":{"total":95927926784,"available":95100665856,"internally_reserved":421888,"allocated":95101087744,"data_stored":95101087744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":864564,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]}]},{"osd":5,"up_from":21,"seq":90194313227,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":813484,"kb_used_data":92867364,"kb_used_omap":858,"kb_used_meta":0,"kb_avail":92866132,"statfs":{"total":95927926784,"available":95094919168,"internally_reserved":1261568,"allocated":95096180736,"data_stored":95096180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":878683,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":6,"up_from":21,"seq":90194313227,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":814908,"kb_used_data":92866648,"kb_used_omap":1025,"kb_used_meta":0,"kb_avail":92864708,"statfs":{"total":95927926784,"available":95093460992,"internally_reserved":1986560,"allocated":95095447552,"data_stored":95095447552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1049737,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.353}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]}]},{"osd":7,"up_from":21,"seq":90194313228,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":802964,"kb_used_data":92876892,"kb_used_omap":953,"kb_used_meta":0,"kb_avail":92876652,"statfs":{"total":95927926784,"available":95105691648,"internally_reserved":245760,"allocated":95105937408,"data_stored":95105937408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":976577,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":8,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":814816,"kb_used_data":92866008,"kb_used_omap":1054,"kb_used_meta":0,"kb_avail":92864800,"statfs":{"total":95927926784,"available":95093555200,"internally_reserved":1236992,"allocated":95094792192,"data_stored":95094792192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1080139,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313227,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":811304,"kb_used_data":92869552,"kb_used_omap":1168,"kb_used_meta":0,"kb_avail":92868312,"statfs":{"total":95927926784,"available":95097151488,"internally_reserved":1269760,"allocated":95098421248,"data_stored":95098421248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1196891,"internal_metadata":0},"hb_peers":[0,1,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":11,"up_from":21,"seq":90194313227,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":813564,"kb_used_data":92868588,"kb_used_omap":706,"kb_used_meta":0,"kb_avail":92866052,"statfs":{"total":95927926784,"available":95094837248,"internally_reserved":2596864,"allocated":95097434112,"data_stored":95097434112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":723698,"internal_metadata":0},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":12,"up_from":21,"seq":90194313228,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":802340,"kb_used_data":92877380,"kb_used_omap":605,"kb_used_meta":0,"kb_avail":92877276,"statfs":{"total":95927926784,"available":95106330624,"internally_reserved":106496,"allocated":95106437120,"data_stored":95106437120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":619768,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":1,"up_from":21,"seq":90194313228,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":812392,"kb_used_data":92869136,"kb_used_omap":1304,"kb_used_meta":0,"kb_avail":92867224,"statfs":{"total":95927926784,"available":95096037376,"internally_reserved":1957888,"allocated":95097995264,"data_stored":95097995264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1335861,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]}]}],"pool_statfs":[]}} 2024-09-15T01:33:02.326 INFO:tasks.ceph.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-15T01:33:02.326 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:02.318+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:02.420 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:02.419+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:02.464 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:02.464 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:02.464 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:33:02.464 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:02.473 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:33:02.521 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:02.520+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:02.613 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:02.613 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:02.613 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:33:02.613 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:33:02.614 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:02.622 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:33:02.622 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:02.620+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:02.723 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:02.721+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:02.764 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:02.765 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:33:02.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:33:02.765 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:33:02.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.765 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.7s3 34/31 ECSubWrite(tid=17, reqid=client.4477.0:77, at_version=34'6, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:33:02.765 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:33:02.765 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.546263420999999, 2024-09-15T01:33:02.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015250150000000001, 2024-09-15T01:33:02.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.765 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:33:02.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:33:02.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215518+0000", 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3567000000000001e-05 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230278+0000", 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014760300000000001 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230285+0000", 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.285e-06 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230359+0000", 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4025999999999999e-05 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230367+0000", 2024-09-15T01:33:02.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4500000000000004e-06 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230457+0000", 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0457999999999993e-05 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230498+0000", 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0204000000000003e-05 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230554+0000", 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6097999999999997e-05 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230573+0000", 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9252000000000001e-05 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230754+0000", 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018093300000000001 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.770 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.771 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.771 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:90 3.6s0 3:65d6c4b7:::smithi17529310-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:02.771 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:33:02.771 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.535625502999999, 2024-09-15T01:33:02.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.964412163, 2024-09-15T01:33:02.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:02.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:02.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:02.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 90 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226144+0000", 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0490000000000002e-06 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226152+0000", 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5700000000000004e-06 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226157+0000", 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0069999999999997e-06 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.227618+0000", 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014613 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.227698+0000", 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9518000000000007e-05 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.228785+0000", 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010872729999999999 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.228833+0000", 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.85e-05 2024-09-15T01:33:02.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:37.179967+0000", 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.95113379099999995 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:37.179971+0000", 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7620000000000001e-06 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:38.180851+0000", 2024-09-15T01:33:02.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0008801570000001 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:38.180855+0000", 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.416e-06 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.187397+0000", 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.006542346 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.187673+0000", 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000275825 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.190554+0000", 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028811650000000002 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.7s3 34/31 ECSubRead(tid=22, to_read={3:e6e8875f:::smithi17529310-31:head=0,4096,0}, subchunks={3:e6e8875f:::smithi17529310-31:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:02.778 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:33:02.779 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.528253458000002, 2024-09-15T01:33:02.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.074844447999999994, 2024-09-15T01:33:02.779 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.779 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.779 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:33:02.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4300000000000003e-07 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233521+0000", 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0280000000000003e-06 2024-09-15T01:33:02.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308171+0000", 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.074650066000000001 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308178+0000", 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5810000000000002e-06 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308258+0000", 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9977999999999999e-05 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308358+0000", 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100152 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:196 3.6s0 3:65d6c4b7:::smithi17529310-30:head [append 0~0] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:02.782 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:33:02.783 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.558605020999998, 2024-09-15T01:33:02.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007181882, 2024-09-15T01:33:02.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:02.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:02.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:02.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 196 2024-09-15T01:33:02.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.783 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203168+0000", 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5620000000000003e-06 2024-09-15T01:33:02.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203175+0000", 2024-09-15T01:33:02.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5599999999999996e-06 2024-09-15T01:33:02.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203194+0000", 2024-09-15T01:33:02.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8757999999999999e-05 2024-09-15T01:33:02.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203201+0000", 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7429999999999996e-06 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203414+0000", 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021300600000000001 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203419+0000", 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1420000000000001e-06 2024-09-15T01:33:02.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.204700+0000", 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.001280647 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.204945+0000", 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024550199999999998 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.205894+0000", 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094873799999999997 2024-09-15T01:33:02.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206188+0000", 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029375700000000001 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206257+0000", 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9632e-05 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206371+0000", 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011411100000000001 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206444+0000", 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2355999999999996e-05 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210281+0000", 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038376740000000001 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210344+0000", 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2694000000000007e-05 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:197 3.6s0 3:65d6c4b7:::smithi17529310-30:head [rollback 3] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.558551427000001, 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0090962939999999, 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:02.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 197 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8299999999999997e-07 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203220+0000", 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0620000000000002e-06 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203229+0000", 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5329999999999994e-06 2024-09-15T01:33:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203234+0000", 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000002e-06 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203537+0000", 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030286700000000001 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203539+0000", 2024-09-15T01:33:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.277e-06 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206748+0000", 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.003209102 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206862+0000", 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011395000000000001 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207832+0000", 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097048299999999996 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207937+0000", 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104583 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207956+0000", 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9381e-05 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208001+0000", 2024-09-15T01:33:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4642999999999999e-05 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209901+0000", 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001899766 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211474+0000", 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015733190000000001 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212312+0000", 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083790500000000001 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:198 3.6s0 3:65d6c4b7:::smithi17529310-30:head [append 0~0] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.558267234999999, 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.009021902, 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 198 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203501+0000", 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4600000000000004e-07 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203505+0000", 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3830000000000002e-06 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203562+0000", 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7120000000000002e-05 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203570+0000", 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203656+0000", 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6273000000000006e-05 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203658+0000", 2024-09-15T01:33:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4360000000000001e-06 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208296+0000", 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004637574 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208407+0000", 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110994 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209048+0000", 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064081900000000005 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209141+0000", 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3827999999999997e-05 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209164+0000", 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2201e-05 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209206+0000", 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2021999999999998e-05 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209910+0000", 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070434399999999998 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211939+0000", 2024-09-15T01:33:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020290159999999998 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212522+0000", 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058283599999999999 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:211 3.6s0 3:6ba0b44a:::smithi17529310-6:head [copy-from ver 0] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:33:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.563786707999999, 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.22039597699999999, 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197982+0000", 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.618e-06 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197988+0000", 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4519999999999996e-06 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198027+0000", 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9492000000000001e-05 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198036+0000", 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3950000000000007e-06 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198736+0000", 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00069949200000000002 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198740+0000", 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3470000000000001e-06 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210425+0000", 2024-09-15T01:33:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011684905000000001 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210622+0000", 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019688099999999999 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418376+0000", 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.20775439500000001 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:212 3.6s0 3:6ba0b44a:::smithi17529310-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.563723059000001, 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.22052332099999999, 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 212 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198045+0000", 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6499999999999998e-07 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198049+0000", 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4150000000000001e-06 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198058+0000", 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9830000000000002e-06 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198064+0000", 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6219999999999997e-06 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198782+0000", 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071748199999999997 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198783+0000", 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9400000000000001e-06 2024-09-15T01:33:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210808+0000", 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012024217 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210861+0000", 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2859999999999999e-05 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418443+0000", 2024-09-15T01:33:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.20758253099999999 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418509+0000", 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5516000000000001e-05 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418567+0000", 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.889e-05 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs6 39/31 ECSubWrite(tid=56, reqid=osd.11.0:56, at_version=39'21, trim_to=0'0, roll_forward_to=38'17))", 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.518974307000001, 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038235071000000002, 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:33:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242794+0000", 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8599999999999997e-07 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242817+0000", 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3655e-05 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280383+0000", 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037566100999999998 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280390+0000", 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280442+0000", 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1650000000000002e-05 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280448+0000", 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7479999999999999e-06 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280660+0000", 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021205900000000001 2024-09-15T01:33:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280667+0000", 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4710000000000001e-06 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280705+0000", 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7511000000000001e-05 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280722+0000", 2024-09-15T01:33:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7646999999999999e-05 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281028+0000", 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030575099999999999 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s1 44/31 ECSubWrite(tid=29, reqid=osd.5.0:29, at_version=44'21, trim_to=0'0, roll_forward_to=40'19))", 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.240231+0000", 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.52153641, 2024-09-15T01:33:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0092854300000000008, 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240231+0000", 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240231+0000", 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240231+0000", 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3200000000000001e-07 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240255+0000", 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3978999999999999e-05 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240354+0000", 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8795000000000005e-05 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240361+0000", 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7109999999999998e-06 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240412+0000", 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1104999999999999e-05 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.240422+0000", 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0603e-05 2024-09-15T01:33:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241395+0000", 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097287000000000003 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241408+0000", 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.273e-05 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241441+0000", 2024-09-15T01:33:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3243e-05 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241457+0000", 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6324000000000002e-05 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249516+0000", 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080588380000000005 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs2 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.519452234999999, 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011429094000000001, 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242316+0000", 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2900000000000003e-07 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242337+0000", 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1263999999999998e-05 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242392+0000", 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5408999999999998e-05 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242399+0000", 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.866e-06 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242456+0000", 2024-09-15T01:33:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-05 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242466+0000", 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0015e-05 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242685+0000", 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021821399999999999 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242713+0000", 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8589e-05 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242754+0000", 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1173e-05 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242790+0000", 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5167999999999999e-05 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253744+0000", 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010954527 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs1 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.518955851000001, 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010913366000000001, 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242812+0000", 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:33:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242834+0000", 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1827999999999998e-05 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242901+0000", 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6840000000000004e-05 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242908+0000", 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5440000000000001e-06 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242959+0000", 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1057000000000003e-05 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242969+0000", 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8030000000000008e-06 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243189+0000", 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021987499999999999 2024-09-15T01:33:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243212+0000", 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2569999999999999e-05 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243258+0000", 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6452000000000002e-05 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243274+0000", 2024-09-15T01:33:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6426000000000002e-05 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253725+0000", 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010450249 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs6 44/31 ECSubWrite(tid=82, reqid=client.4477.0:281, at_version=44'36, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.518202768999998, 2024-09-15T01:33:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011937237, 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243565+0000", 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0500000000000003e-07 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243585+0000", 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0313e-05 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243689+0000", 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103977 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243696+0000", 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0489999999999998e-06 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243791+0000", 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4394000000000005e-05 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243800+0000", 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2439999999999999e-06 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244285+0000", 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000484802 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244313+0000", 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8515000000000001e-05 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244359+0000", 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5389000000000002e-05 2024-09-15T01:33:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244375+0000", 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6699000000000001e-05 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.255502+0000", 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011126250000000001 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s1 44/31 ECSubWrite(tid=30, reqid=osd.5.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243919+0000", 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.517848147999999, 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010421095999999999, 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243919+0000", 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243919+0000", 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243920+0000", 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2399999999999997e-07 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243929+0000", 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8170000000000005e-06 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245823+0000", 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001894792 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245831+0000", 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9160000000000001e-06 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245883+0000", 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789000000000002e-05 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245894+0000", 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0658e-05 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246000+0000", 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105809 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246019+0000", 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250999999999999e-05 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246056+0000", 2024-09-15T01:33:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6636e-05 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249628+0000", 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035721339999999998 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.254340+0000", 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047124699999999999 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.9s2 44/31 ECSubWrite(tid=64, reqid=osd.14.0:60, at_version=0'0, trim_to=0'0, roll_forward_to=44'18))", 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.245393+0000", 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.516373763000001, 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087638879999999992, 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245393+0000", 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245393+0000", 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245394+0000", 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3900000000000003e-07 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245406+0000", 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2092e-05 2024-09-15T01:33:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245442+0000", 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5505e-05 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245454+0000", 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1549000000000001e-05 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245508+0000", 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4781999999999998e-05 2024-09-15T01:33:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245519+0000", 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0497e-05 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245624+0000", 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105402 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245646+0000", 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1739e-05 2024-09-15T01:33:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245687+0000", 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0728e-05 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249592+0000", 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039053099999999999 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.254157+0000", 2024-09-15T01:33:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045653450000000002 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 44/31 ECSubWrite(tid=36, reqid=osd.7.0:36, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.246398+0000", 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.515368755000001, 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0094625609999999995, 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246398+0000", 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246398+0000", 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246399+0000", 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8e-07 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246422+0000", 2024-09-15T01:33:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3703e-05 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247337+0000", 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091461400000000001 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247344+0000", 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1570000000000003e-06 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247394+0000", 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9966000000000002e-05 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247403+0000", 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7260000000000008e-06 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247834+0000", 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043114100000000001 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:33:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5984000000000001e-05 2024-09-15T01:33:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247900+0000", 2024-09-15T01:33:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9941999999999998e-05 2024-09-15T01:33:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.250749+0000", 2024-09-15T01:33:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028491440000000001 2024-09-15T01:33:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.255861+0000", 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051120039999999999 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 44/31 ECSubWrite(tid=39, reqid=client.4477.0:283, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.493480778999999, 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012672954, 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268287+0000", 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.85e-07 2024-09-15T01:33:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268806+0000", 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00051859699999999998 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272018+0000", 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032124620000000001 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272032+0000", 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4008e-05 2024-09-15T01:33:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272076+0000", 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3807999999999998e-05 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272085+0000", 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9749999999999996e-06 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272331+0000", 2024-09-15T01:33:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024564600000000003 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272349+0000", 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7988000000000001e-05 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272412+0000", 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2718000000000005e-05 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272426+0000", 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.280959+0000", 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085329199999999994 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:293 3.6s0 3:65d6c4b7:::smithi17529310-30:head [create,setxattr dgeSiblb`YZWoafOlamThlmYtW[NsgYatRkY]Rkn (17332) in=40b,setxattr c`hS`asrnTY^]RZfhhpk^WaRiegkSORj]tZRqnbU (945) in=40b,setxattr biPi^V_QgOq`^qVaOnjr`odUZpilfdNcYZaf]N]q (594) in=40b,setxattr ilZ^tXpioaqcTlZtZR[RiVbNk[cWS^Q`oYjaj_j[ (17824) in=40b,setxattr _p_ks]sbj_gRlVjZr]OjWXZ]gf (1718) in=26b,setxattr eQ\\\\cjZhQZanRhQZcRjiTbSiomgf]n`[kNbokP^` (4560) in=40b,setxattr gqXgeRc[[^pjV]NraefeRN]mbW\\eSlcSOXdPrmSb (1823) in=40b,setxattr Y]hO^[akiVr`]YhQ[oTnksOSSdslk[h\\idalij_d (701) in=40b,setxattr VqPasiXWWe^eT\\Q`e`__hRmg]O_kYeg_mOrmbaaN (1051) in=40b,setxattr ]]i\\^cfhf_mbRNmfWcpO^Tf[Sa`abbsR_el\\OTgR (1366) in=40b,setxattr e]a_`g\\Of\\ipZOgP]aadXoRp\\q^kh_cpolQ[RjXf (1402) in=40b,setxattr VTaQUZ_lrfT`VmUqPmbhXZciq[kkhXT`nkTblbh` (1298) in=40b,setxattr aUjlO\\kVcZYT^ZU\\jbkfd_VRRa]nqn_[T\\ZhlZp[ (14664) in=40b,setxattr sZScfemtX`pkqf^`dZ]ap (1718) in=21b,setxattr agej`oOViShjhSZSRaYRa_k`gk^ecQqfqgOjpa[q (33) in=40b,setxattr fsad_S`\\kPa[[i[VVsaQOZXpPa]pp\\fOQ`tbY`Qg (13472) in=40b,setxattr c\\g]ORfZrTgatTVmX^dSZtTYaXN^YVoZX[Sajonf (1928) in=40b,setxattr T`^]hVNaYYjRdaNne[XsYdaqR]SSTVTsmUnWXqnp (911) in=40b,setxattr ]rWN_b^TgarYNoqUP]qjm_kNbNiPbpth^PrblPjb (315) in=40b,setxattr XtYoarpQjhPgloXSNSob_agQ\\WY\\\\nThcs[XqPZP (1857) in=40b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.269581+0000", 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.492186553, 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088342349999999993, 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 293 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269581+0000", 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269581+0000", 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269583+0000", 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269783+0000", 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020008499999999999 2024-09-15T01:33:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269799+0000", 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.611e-05 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269805+0000", 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1739999999999997e-06 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269863+0000", 2024-09-15T01:33:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8013000000000002e-05 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269986+0000", 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012216500000000001 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271038+0000", 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001052313 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271289+0000", 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025120200000000001 2024-09-15T01:33:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271335+0000", 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5512000000000001e-05 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271424+0000", 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9284999999999996e-05 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.271758+0000", 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033430299999999999 2024-09-15T01:33:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.276945+0000", 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051865039999999998 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.278415+0000", 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001470138 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s1 55/31 ECSubWrite(tid=67, reqid=client.4477.0:503, at_version=0'0, trim_to=0'0, roll_forward_to=55'51))", 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.106011+0000", 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.6557563670000004, 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017261852000000001, 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.106011+0000", 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.106011+0000", 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.106013+0000", 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2079999999999999e-06 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.106037+0000", 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4114e-05 2024-09-15T01:33:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122822+0000", 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016784701999999999 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122834+0000", 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2055000000000001e-05 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122891+0000", 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7244999999999998e-05 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122899+0000", 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122994+0000", 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4554000000000003e-05 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.123004+0000", 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.049e-05 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.123055+0000", 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0927999999999998e-05 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.123074+0000", 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9177999999999998e-05 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.123273+0000", 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019858899999999999 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 60/38 ECSubWriteReply(tid=51, last_complete=60'31, committed=1, applied=1))", 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:02.100140+0000", 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.66162697800000003, 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.402923223, 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.100140+0000", 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.100140+0000", 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.100141+0000", 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9599999999999999e-07 2024-09-15T01:33:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.100145+0000", 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1949999999999996e-06 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.502953+0000", 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.40280827200000002 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.502961+0000", 2024-09-15T01:33:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9840000000000001e-06 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.503036+0000", 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4394999999999995e-05 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.503063+0000", 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7781e-05 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:02.881 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:02.881 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:33:02.883 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:33:02.883 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:02.880+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:02.983 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:02.982+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:03.026 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:03.026 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:03.026 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:33:03.026 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:03.035 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:33:03.052 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:03.052 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:03.073 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:03.084 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:03.082+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:03.087 INFO:tasks.rados.rados.0.smithi175.stdout:213: done (2 left) 2024-09-15T01:33:03.087 INFO:tasks.rados.rados.0.smithi175.stdout:215: done (1 left) 2024-09-15T01:33:03.087 INFO:tasks.rados.rados.0.smithi175.stdout:216: done (0 left) 2024-09-15T01:33:03.087 INFO:tasks.rados.rados.0.smithi175.stdout:217: rollback oid 11 current snap is 15 2024-09-15T01:33:03.087 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 11 to 13 2024-09-15T01:33:03.087 INFO:tasks.rados.rados.0.smithi175.stdout:218: read oid 37 snap -1 2024-09-15T01:33:03.087 INFO:tasks.rados.rados.0.smithi175.stdout:218: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-15T01:33:03.088 INFO:tasks.rados.rados.0.smithi175.stdout:219: copy_from oid 21 from oid 22 current snap is 15 2024-09-15T01:33:03.088 INFO:tasks.rados.rados.0.smithi175.stdout:220: snap_remove snap 14 2024-09-15T01:33:03.100 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:33:03.100 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:03.120 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:03.139 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:03.158 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:03.158 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:03.159 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:03.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:33:03.179 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:03.180 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:03.199 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:03.199 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:03.199 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:03.198+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:03.220 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:03.220 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:03.240 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:03.240 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:03.261 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:03.262 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:03.283 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:03.283 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:03.284 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:03.285 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:03.285 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:03.282+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:03.294 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:33:03.302 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:03.324 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:03.324 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:03.345 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:03.346 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:03.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:03.344+0000 7fe1e37fe640 0 -- 172.21.15.42:0/3443778940 >> [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] conn(0x7fe1e4195f20 msgr2=0x7fe1e4198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:03.386 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:03.385+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:03.404 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:03.404 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:03.413 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:33:03.487 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:03.485+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:03.553 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:03.553 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:03.562 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:33:03.566 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:03.618 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:03.617+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:03.663 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:03.664 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:03.672 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-15T01:33:03.689 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:03.687+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:03.789 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:03.787+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:03.813 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:03.813 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:03.822 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-15T01:33:03.890 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:03.889+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:03.922 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:03.922 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:03.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-15T01:33:03.991 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:03.989+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:04.040 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:33:04.088 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-15T01:33:04.090 INFO:tasks.rados.rados.0.smithi175.stdout:220: done (3 left) 2024-09-15T01:33:04.091 INFO:tasks.rados.rados.0.smithi175.stdout:221: rollback oid 2 current snap is 15 2024-09-15T01:33:04.091 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 2 to 10 2024-09-15T01:33:04.091 INFO:tasks.rados.rados.0.smithi175.stdout:217: finishing rollback tid 0 to smithi17529310-11 2024-09-15T01:33:04.091 INFO:tasks.rados.rados.0.smithi175.stdout:217: finishing rollback tid 1 to smithi17529310-11 2024-09-15T01:33:04.091 INFO:tasks.rados.rados.0.smithi175.stdout:218: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-15T01:33:04.092 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:04.090+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:04.108 INFO:tasks.rados.rados.0.smithi175.stdout:219: finishing copy_from to smithi17529310-21 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:219: got expected ENOENT (src dne) 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:219: finishing copy_from racing read to smithi17529310-21 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:217: finishing rollback tid 2 to smithi17529310-11 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 11 v 12 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:221: finishing rollback tid 0 to smithi17529310-2 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:221: finishing rollback tid 1 to smithi17529310-2 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:221: finishing rollback tid 2 to smithi17529310-2 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 2 v 101 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:217: done (3 left) 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:218: done (2 left) 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:219: done (1 left) 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:221: done (0 left) 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:222: read oid 26 snap -1 2024-09-15T01:33:04.109 INFO:tasks.rados.rados.0.smithi175.stdout:222: expect (ObjNum 103 snap 10 seq_num 103) 2024-09-15T01:33:04.110 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 1 current snap is 15 2024-09-15T01:33:04.110 INFO:tasks.rados.rados.0.smithi175.stdout:223: seq_num 112 ranges {0=720896,720896=802816,1523712=360448} 2024-09-15T01:33:04.112 INFO:tasks.rados.rados.0.smithi175.stdout:223: writing smithi17529310-1 from 0 to 720896 tid 1 2024-09-15T01:33:04.118 INFO:tasks.rados.rados.0.smithi175.stdout:223: writing smithi17529310-1 from 720896 to 1523712 tid 2 2024-09-15T01:33:04.120 INFO:tasks.rados.rados.0.smithi175.stdout:223: writing smithi17529310-1 from 1523712 to 1884160 tid 3 2024-09-15T01:33:04.120 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 30 current snap is 15 2024-09-15T01:33:04.120 INFO:tasks.rados.rados.0.smithi175.stdout:224: seq_num 113 ranges {1998848=737280,2736128=557056,3293184=606208} 2024-09-15T01:33:04.136 INFO:tasks.rados.rados.0.smithi175.stdout:224: writing smithi17529310-30 from 1998848 to 2736128 tid 1 2024-09-15T01:33:04.138 INFO:tasks.rados.rados.0.smithi175.stdout:224: writing smithi17529310-30 from 2736128 to 3293184 tid 2 2024-09-15T01:33:04.143 INFO:tasks.rados.rados.0.smithi175.stdout:224: writing smithi17529310-30 from 3293184 to 3899392 tid 3 2024-09-15T01:33:04.143 INFO:tasks.rados.rados.0.smithi175.stdout:225: snap_create 2024-09-15T01:33:04.143 INFO:tasks.rados.rados.0.smithi175.stdout:223: finishing write tid 1 to smithi17529310-1 2024-09-15T01:33:04.143 INFO:tasks.rados.rados.0.smithi175.stdout:223: finishing write tid 2 to smithi17529310-1 2024-09-15T01:33:04.143 INFO:tasks.rados.rados.0.smithi175.stdout:223: finishing write tid 3 to smithi17529310-1 2024-09-15T01:33:04.143 INFO:tasks.rados.rados.0.smithi175.stdout:223: finishing write tid 4 to smithi17529310-1 2024-09-15T01:33:04.143 INFO:tasks.rados.rados.0.smithi175.stdout:223: finishing write tid 5 to smithi17529310-1 2024-09-15T01:33:04.143 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 1 v 27 (ObjNum 112 snap 15 seq_num 112) dirty exists 2024-09-15T01:33:04.143 INFO:tasks.rados.rados.0.smithi175.stdout:223: left oid 1 (ObjNum 112 snap 15 seq_num 112) 2024-09-15T01:33:04.143 INFO:tasks.rados.rados.0.smithi175.stdout:222: expect (ObjNum 103 snap 10 seq_num 103) 2024-09-15T01:33:04.158 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:04.158 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:04.167 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:33:04.193 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:04.191+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:04.206 INFO:tasks.rados.rados.0.smithi175.stdout:224: finishing write tid 1 to smithi17529310-30 2024-09-15T01:33:04.206 INFO:tasks.rados.rados.0.smithi175.stdout:224: finishing write tid 2 to smithi17529310-30 2024-09-15T01:33:04.206 INFO:tasks.rados.rados.0.smithi175.stdout:224: finishing write tid 3 to smithi17529310-30 2024-09-15T01:33:04.206 INFO:tasks.rados.rados.0.smithi175.stdout:224: finishing write tid 4 to smithi17529310-30 2024-09-15T01:33:04.206 INFO:tasks.rados.rados.0.smithi175.stdout:224: finishing write tid 5 to smithi17529310-30 2024-09-15T01:33:04.206 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 30 v 36 (ObjNum 113 snap 15 seq_num 113) dirty exists 2024-09-15T01:33:04.206 INFO:tasks.rados.rados.0.smithi175.stdout:224: left oid 30 (ObjNum 113 snap 15 seq_num 113) 2024-09-15T01:33:04.264 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:04.265 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:04.274 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:33:04.317 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:04.315+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:04.370 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:04.370 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:04.379 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:33:04.395 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:04.393+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:04.495 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:04.493+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:04.516 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:04.516 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:04.525 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:33:04.597 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:04.595+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:04.660 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:04.661 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:04.668 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:33:04.712 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:04.710+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:04.765 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:04.765 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:04.773 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:33:04.798 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:04.796+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:04.900 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:04.898+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:04.910 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:04.910 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:04.919 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:33:05.000 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:04.999+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:05.014 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:05.014 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:05.022 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:33:05.094 INFO:tasks.rados.rados.0.smithi175.stdout:222: done (3 left) 2024-09-15T01:33:05.095 INFO:tasks.rados.rados.0.smithi175.stdout:223: done (2 left) 2024-09-15T01:33:05.095 INFO:tasks.rados.rados.0.smithi175.stdout:224: done (1 left) 2024-09-15T01:33:05.095 INFO:tasks.rados.rados.0.smithi175.stdout:225: done (0 left) 2024-09-15T01:33:05.095 INFO:tasks.rados.rados.0.smithi175.stdout:226: copy_from oid 40 from oid 13 current snap is 16 2024-09-15T01:33:05.095 INFO:tasks.rados.rados.0.smithi175.stdout:227: copy_from oid 38 from oid 48 current snap is 16 2024-09-15T01:33:05.095 INFO:tasks.rados.rados.0.smithi175.stdout:228: setattr oid 37 current snap is 16 2024-09-15T01:33:05.096 INFO:tasks.rados.rados.0.smithi175.stdout:229: rmattr oid 44 current snap is 16 2024-09-15T01:33:05.096 INFO:tasks.rados.rados.0.smithi175.stdout:229: done (3 left) 2024-09-15T01:33:05.096 INFO:tasks.rados.rados.0.smithi175.stdout:230: snap_create 2024-09-15T01:33:05.101 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:05.099+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:05.102 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 37 v 33 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-15T01:33:05.106 INFO:tasks.rados.rados.0.smithi175.stdout:226: finishing copy_from racing read to smithi17529310-40 2024-09-15T01:33:05.108 INFO:tasks.rados.rados.0.smithi175.stdout:226: finishing copy_from to smithi17529310-40 2024-09-15T01:33:05.108 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 40 v 34 (ObjNum 74 snap 5 seq_num 74) dirty exists 2024-09-15T01:33:05.122 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:05.122 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:05.124 INFO:tasks.rados.rados.0.smithi175.stdout:227: finishing copy_from racing read to smithi17529310-38 2024-09-15T01:33:05.131 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-15T01:33:05.137 INFO:tasks.rados.rados.0.smithi175.stdout:227: finishing copy_from to smithi17529310-38 2024-09-15T01:33:05.137 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 38 v 54 (ObjNum 84 snap 7 seq_num 84) dirty exists 2024-09-15T01:33:05.202 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:05.199+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:05.228 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:05.228 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:05.236 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-15T01:33:05.320 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:05.318+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:05.372 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:05.372 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:05.380 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-15T01:33:05.404 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:05.402+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:05.505 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:05.503+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:05.518 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:05.518 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:05.526 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:33:05.578 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:33:05.606 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:05.604+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:05.636 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:05.636 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:05.645 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:33:05.707 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:05.705+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:05.718 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:05.718 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:05.737 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:05.742 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:05.743 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:05.751 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:33:05.757 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:05.757 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:05.778 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:05.778 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:05.796 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:05.796 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:05.807 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:05.805+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:05.816 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:05.816 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:05.834 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:05.834 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:05.851 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:05.852 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:05.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:05.852 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:05.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.852 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=152, reqid=osd.13.0:152, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:05.852 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.243665+0000", 2024-09-15T01:33:05.852 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.603657794, 2024-09-15T01:33:05.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019478882999999999, 2024-09-15T01:33:05.853 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.853 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.853 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243665+0000", 2024-09-15T01:33:05.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243665+0000", 2024-09-15T01:33:05.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243666+0000", 2024-09-15T01:33:05.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.34e-07 2024-09-15T01:33:05.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243686+0000", 2024-09-15T01:33:05.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9830000000000002e-05 2024-09-15T01:33:05.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243778+0000", 2024-09-15T01:33:05.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2367999999999994e-05 2024-09-15T01:33:05.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243802+0000", 2024-09-15T01:33:05.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3646e-05 2024-09-15T01:33:05.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243862+0000", 2024-09-15T01:33:05.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0579999999999999e-05 2024-09-15T01:33:05.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243876+0000", 2024-09-15T01:33:05.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4025e-05 2024-09-15T01:33:05.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245801+0000", 2024-09-15T01:33:05.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0019244399999999999 2024-09-15T01:33:05.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246115+0000", 2024-09-15T01:33:05.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00031487699999999997 2024-09-15T01:33:05.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248315+0000", 2024-09-15T01:33:05.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021992140000000001 2024-09-15T01:33:05.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248329+0000", 2024-09-15T01:33:05.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4450000000000001e-05 2024-09-15T01:33:05.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263144+0000", 2024-09-15T01:33:05.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014815218999999999 2024-09-15T01:33:05.863 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.863 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.863 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.864 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:05.864 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:33:05.864 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.600355191999999, 2024-09-15T01:33:05.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019581815999999998, 2024-09-15T01:33:05.864 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.864 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.864 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:33:05.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:33:05.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246969+0000", 2024-09-15T01:33:05.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7000000000000003e-07 2024-09-15T01:33:05.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247008+0000", 2024-09-15T01:33:05.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9354999999999998e-05 2024-09-15T01:33:05.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247051+0000", 2024-09-15T01:33:05.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2849000000000003e-05 2024-09-15T01:33:05.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247060+0000", 2024-09-15T01:33:05.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4589999999999998e-06 2024-09-15T01:33:05.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248680+0000", 2024-09-15T01:33:05.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001620058 2024-09-15T01:33:05.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248690+0000", 2024-09-15T01:33:05.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8660000000000001e-06 2024-09-15T01:33:05.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248945+0000", 2024-09-15T01:33:05.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025509900000000001 2024-09-15T01:33:05.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251105+0000", 2024-09-15T01:33:05.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021604290000000002 2024-09-15T01:33:05.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251177+0000", 2024-09-15T01:33:05.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1680999999999994e-05 2024-09-15T01:33:05.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251189+0000", 2024-09-15T01:33:05.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2661e-05 2024-09-15T01:33:05.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266550+0000", 2024-09-15T01:33:05.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015360389 2024-09-15T01:33:05.873 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.873 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.874 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.874 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=28, reqid=osd.8.0:28, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:05.874 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:05.874 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.597173321, 2024-09-15T01:33:05.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022199446000000001, 2024-09-15T01:33:05.875 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.875 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.875 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:05.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:05.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:05.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4700000000000001e-07 2024-09-15T01:33:05.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250167+0000", 2024-09-15T01:33:05.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7384000000000002e-05 2024-09-15T01:33:05.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250202+0000", 2024-09-15T01:33:05.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4912e-05 2024-09-15T01:33:05.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250210+0000", 2024-09-15T01:33:05.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6029999999999999e-06 2024-09-15T01:33:05.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250258+0000", 2024-09-15T01:33:05.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8109999999999998e-05 2024-09-15T01:33:05.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250265+0000", 2024-09-15T01:33:05.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4400000000000002e-06 2024-09-15T01:33:05.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250466+0000", 2024-09-15T01:33:05.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000201227 2024-09-15T01:33:05.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251113+0000", 2024-09-15T01:33:05.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00064769299999999997 2024-09-15T01:33:05.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251179+0000", 2024-09-15T01:33:05.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5895999999999994e-05 2024-09-15T01:33:05.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251196+0000", 2024-09-15T01:33:05.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6810999999999999e-05 2024-09-15T01:33:05.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272349+0000", 2024-09-15T01:33:05.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021153222999999999 2024-09-15T01:33:05.887 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.888 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.888 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.888 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:05.889 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:33:05.889 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.595819193000001, 2024-09-15T01:33:05.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023186075, 2024-09-15T01:33:05.889 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.889 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.889 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:33:05.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:33:05.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251505+0000", 2024-09-15T01:33:05.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.09e-07 2024-09-15T01:33:05.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251531+0000", 2024-09-15T01:33:05.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5834999999999999e-05 2024-09-15T01:33:05.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251574+0000", 2024-09-15T01:33:05.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2879000000000001e-05 2024-09-15T01:33:05.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251580+0000", 2024-09-15T01:33:05.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8460000000000002e-06 2024-09-15T01:33:05.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251639+0000", 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8331000000000001e-05 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251649+0000", 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9930000000000007e-06 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251890+0000", 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00024173400000000001 2024-09-15T01:33:05.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251923+0000", 2024-09-15T01:33:05.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2607999999999997e-05 2024-09-15T01:33:05.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251985+0000", 2024-09-15T01:33:05.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2238999999999998e-05 2024-09-15T01:33:05.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251998+0000", 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2979999999999999e-05 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274690+0000", 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022691821000000001 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s1 44/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=44'26))", 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.594602195, 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019696631999999999, 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252722+0000", 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3099999999999996e-07 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252734+0000", 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1576999999999999e-05 2024-09-15T01:33:05.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252780+0000", 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6820999999999998e-05 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252788+0000", 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2710000000000003e-06 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252848+0000", 2024-09-15T01:33:05.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0718999999999999e-05 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252859+0000", 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0342e-05 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252960+0000", 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000101741 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252970+0000", 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0008e-05 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253967+0000", 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00099612599999999996 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253987+0000", 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0064e-05 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018431032 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=154, reqid=client.4477.0:282, at_version=44'19, trim_to=0'0, roll_forward_to=44'17))", 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.590590535, 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023652260000000001, 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.903 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256734+0000", 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3900000000000004e-07 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256754+0000", 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0279e-05 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256785+0000", 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0703000000000001e-05 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256798+0000", 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3606e-05 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256843+0000", 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5065000000000003e-05 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256851+0000", 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5580000000000003e-06 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257013+0000", 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016264 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257339+0000", 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032564200000000001 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257402+0000", 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3162000000000003e-05 2024-09-15T01:33:05.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258083+0000", 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00068126099999999995 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280385+0000", 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022301504999999999 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=85, reqid=osd.11.0:84, at_version=0'0, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.590266751000001, 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020078747000000001, 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257057+0000", 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5000000000000002e-07 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257067+0000", 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5540000000000003e-06 2024-09-15T01:33:05.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257097+0000", 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9892000000000001e-05 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257102+0000", 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3360000000000002e-06 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257155+0000", 2024-09-15T01:33:05.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3313000000000003e-05 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257164+0000", 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1060000000000006e-06 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257257+0000", 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2678999999999998e-05 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257341+0000", 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3459000000000002e-05 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257403+0000", 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2477000000000005e-05 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257420+0000", 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7170999999999999e-05 2024-09-15T01:33:05.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277135+0000", 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019715010000000002 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=30, reqid=osd.8.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.589999479999999, 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020096306000000001, 2024-09-15T01:33:05.914 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3099999999999997e-07 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257335+0000", 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0727000000000001e-05 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257371+0000", 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5583999999999997e-05 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257378+0000", 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.754e-06 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257427+0000", 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9048999999999997e-05 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257475+0000", 2024-09-15T01:33:05.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8140999999999999e-05 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257605+0000", 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012981700000000001 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257640+0000", 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.557e-05 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257674+0000", 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4215999999999999e-05 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257692+0000", 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7988000000000001e-05 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277420+0000", 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019727728999999999 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.919 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.920 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=86, reqid=osd.11.0:86, at_version=44'42, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:05.920 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:33:05.920 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.589408226, 2024-09-15T01:33:05.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022807320999999998, 2024-09-15T01:33:05.920 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.920 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.920 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257916+0000", 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257951+0000", 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5754999999999999e-05 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257982+0000", 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0732000000000003e-05 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257987+0000", 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1900000000000003e-06 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258036+0000", 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8547999999999997e-05 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258047+0000", 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1082e-05 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258512+0000", 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00046539599999999997 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258527+0000", 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5121e-05 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258568+0000", 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0305000000000001e-05 2024-09-15T01:33:05.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258581+0000", 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3434999999999999e-05 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280722+0000", 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022141137000000002 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=155, reqid=client.4477.0:282, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.588483429, 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021774615000000001, 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:33:05.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258841+0000", 2024-09-15T01:33:05.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5899999999999995e-07 2024-09-15T01:33:05.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258853+0000", 2024-09-15T01:33:05.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2115e-05 2024-09-15T01:33:05.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258889+0000", 2024-09-15T01:33:05.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6007000000000003e-05 2024-09-15T01:33:05.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258896+0000", 2024-09-15T01:33:05.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:33:05.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258958+0000", 2024-09-15T01:33:05.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2242999999999993e-05 2024-09-15T01:33:05.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258966+0000", 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2370000000000003e-06 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259070+0000", 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000104171 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259104+0000", 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4569000000000001e-05 2024-09-15T01:33:05.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259140+0000", 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5698000000000003e-05 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259155+0000", 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4541e-05 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280614+0000", 2024-09-15T01:33:05.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021459865000000002 2024-09-15T01:33:05.933 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.933 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.933 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.934 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:05.935 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:05.935 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:05.936 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:05.936 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:05.936 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:33:05.936 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.588054054000001, 2024-09-15T01:33:05.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027513342999999999, 2024-09-15T01:33:05.936 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.936 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.936 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:33:05.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:33:05.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259270+0000", 2024-09-15T01:33:05.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.33e-07 2024-09-15T01:33:05.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259295+0000", 2024-09-15T01:33:05.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4413000000000002e-05 2024-09-15T01:33:05.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259335+0000", 2024-09-15T01:33:05.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0154999999999998e-05 2024-09-15T01:33:05.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259344+0000", 2024-09-15T01:33:05.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8890000000000008e-06 2024-09-15T01:33:05.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259414+0000", 2024-09-15T01:33:05.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9929000000000001e-05 2024-09-15T01:33:05.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259423+0000", 2024-09-15T01:33:05.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0289999999999994e-06 2024-09-15T01:33:05.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263376+0000", 2024-09-15T01:33:05.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003953036 2024-09-15T01:33:05.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263393+0000", 2024-09-15T01:33:05.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7444000000000001e-05 2024-09-15T01:33:05.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263481+0000", 2024-09-15T01:33:05.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8251999999999995e-05 2024-09-15T01:33:05.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266784+0000", 2024-09-15T01:33:05.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003303074 2024-09-15T01:33:05.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286783+0000", 2024-09-15T01:33:05.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019998189 2024-09-15T01:33:05.943 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.943 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.943 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.943 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=31, reqid=client.4477.0:288, at_version=44'22, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:05.943 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:05.943 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.583286683000001, 2024-09-15T01:33:05.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022720738000000001, 2024-09-15T01:33:05.943 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.943 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.943 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:05.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:05.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:05.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6599999999999995e-07 2024-09-15T01:33:05.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264398+0000", 2024-09-15T01:33:05.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00036057299999999999 2024-09-15T01:33:05.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264436+0000", 2024-09-15T01:33:05.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7968e-05 2024-09-15T01:33:05.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264444+0000", 2024-09-15T01:33:05.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8949999999999998e-06 2024-09-15T01:33:05.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264489+0000", 2024-09-15T01:33:05.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4954999999999999e-05 2024-09-15T01:33:05.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264499+0000", 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0574000000000001e-05 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266178+0000", 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001678543 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266188+0000", 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0624e-05 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268728+0000", 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025397240000000001 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268912+0000", 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018415400000000001 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286757+0000", 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.017844961999999999 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=32, reqid=client.4477.0:289, at_version=44'23, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.580087245000001, 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019719520000000001, 2024-09-15T01:33:05.949 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267237+0000", 2024-09-15T01:33:05.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3099999999999998e-07 2024-09-15T01:33:05.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267495+0000", 2024-09-15T01:33:05.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025857700000000001 2024-09-15T01:33:05.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267528+0000", 2024-09-15T01:33:05.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2617999999999998e-05 2024-09-15T01:33:05.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267534+0000", 2024-09-15T01:33:05.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0159999999999997e-06 2024-09-15T01:33:05.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267584+0000", 2024-09-15T01:33:05.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0176999999999997e-05 2024-09-15T01:33:05.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.953 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267594+0000", 2024-09-15T01:33:05.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0161e-05 2024-09-15T01:33:05.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.953 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267768+0000", 2024-09-15T01:33:05.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017420000000000001 2024-09-15T01:33:05.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268732+0000", 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00096413399999999995 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268818+0000", 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5583999999999999e-05 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268943+0000", 2024-09-15T01:33:05.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012452399999999999 2024-09-15T01:33:05.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.955 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.955 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286956+0000", 2024-09-15T01:33:05.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018012997999999999 2024-09-15T01:33:05.955 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.955 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.955 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.955 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=91, reqid=osd.11.0:87, at_version=0'0, trim_to=0'0, roll_forward_to=44'44))", 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.579994801000002, 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019815672999999999, 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:33:05.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267329+0000", 2024-09-15T01:33:05.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7500000000000001e-07 2024-09-15T01:33:05.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267339+0000", 2024-09-15T01:33:05.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0351e-05 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267385+0000", 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6008999999999999e-05 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267393+0000", 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7810000000000005e-06 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267443+0000", 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0139000000000002e-05 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267454+0000", 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1012999999999999e-05 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267585+0000", 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013078699999999999 2024-09-15T01:33:05.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268732+0000", 2024-09-15T01:33:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001146768 2024-09-15T01:33:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268817+0000", 2024-09-15T01:33:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5724999999999997e-05 2024-09-15T01:33:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.961 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268838+0000", 2024-09-15T01:33:05.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0954e-05 2024-09-15T01:33:05.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.961 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.961 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.287144+0000", 2024-09-15T01:33:05.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018305770999999998 2024-09-15T01:33:05.961 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.961 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.961 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.962 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=33, reqid=client.4477.0:290, at_version=44'24, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:05.962 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.268479+0000", 2024-09-15T01:33:05.962 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.578843935999998, 2024-09-15T01:33:05.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018839779000000001, 2024-09-15T01:33:05.962 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.962 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.962 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.962 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.962 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268479+0000", 2024-09-15T01:33:05.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.963 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.963 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.963 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268479+0000", 2024-09-15T01:33:05.963 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.963 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.963 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.963 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.963 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268480+0000", 2024-09-15T01:33:05.963 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:33:05.963 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268607+0000", 2024-09-15T01:33:05.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012669499999999999 2024-09-15T01:33:05.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268635+0000", 2024-09-15T01:33:05.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8778000000000001e-05 2024-09-15T01:33:05.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268642+0000", 2024-09-15T01:33:05.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1539999999999999e-06 2024-09-15T01:33:05.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268696+0000", 2024-09-15T01:33:05.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4148999999999999e-05 2024-09-15T01:33:05.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268706+0000", 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9229999999999997e-06 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268871+0000", 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016481300000000001 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268904+0000", 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3188999999999997e-05 2024-09-15T01:33:05.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.967 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.967 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271557+0000", 2024-09-15T01:33:05.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0026537380000000001 2024-09-15T01:33:05.967 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.967 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.967 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271613+0000", 2024-09-15T01:33:05.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5708999999999998e-05 2024-09-15T01:33:05.967 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.967 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.287319+0000", 2024-09-15T01:33:05.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015705909000000001 2024-09-15T01:33:05.968 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.968 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.968 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.968 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=32, reqid=client.4477.0:294, at_version=44'23, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:05.968 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:33:05.968 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.577020588, 2024-09-15T01:33:05.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021391184000000001, 2024-09-15T01:33:05.969 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.969 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.969 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.969 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:33:05.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:33:05.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270304+0000", 2024-09-15T01:33:05.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7499999999999998e-07 2024-09-15T01:33:05.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270326+0000", 2024-09-15T01:33:05.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2821999999999999e-05 2024-09-15T01:33:05.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270366+0000", 2024-09-15T01:33:05.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9708e-05 2024-09-15T01:33:05.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270374+0000", 2024-09-15T01:33:05.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6709999999999999e-06 2024-09-15T01:33:05.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270433+0000", 2024-09-15T01:33:05.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9274999999999997e-05 2024-09-15T01:33:05.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270444+0000", 2024-09-15T01:33:05.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.131e-05 2024-09-15T01:33:05.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.973 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.973 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271974+0000", 2024-09-15T01:33:05.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001529996 2024-09-15T01:33:05.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.973 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.973 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271986+0000", 2024-09-15T01:33:05.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1443e-05 2024-09-15T01:33:05.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272180+0000", 2024-09-15T01:33:05.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019405299999999999 2024-09-15T01:33:05.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272366+0000", 2024-09-15T01:33:05.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018627699999999999 2024-09-15T01:33:05.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.291694+0000", 2024-09-15T01:33:05.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019327654 2024-09-15T01:33:05.975 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.975 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.975 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.976 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:05.976 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:33:05.976 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.576423909999999, 2024-09-15T01:33:05.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024535670999999998, 2024-09-15T01:33:05.976 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.976 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.976 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:33:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:33:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270900+0000", 2024-09-15T01:33:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.23e-07 2024-09-15T01:33:05.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270920+0000", 2024-09-15T01:33:05.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.022e-05 2024-09-15T01:33:05.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270953+0000", 2024-09-15T01:33:05.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3136000000000003e-05 2024-09-15T01:33:05.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270960+0000", 2024-09-15T01:33:05.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4280000000000001e-06 2024-09-15T01:33:05.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272041+0000", 2024-09-15T01:33:05.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010816440000000001 2024-09-15T01:33:05.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272050+0000", 2024-09-15T01:33:05.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5269999999999998e-06 2024-09-15T01:33:05.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272149+0000", 2024-09-15T01:33:05.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9464999999999997e-05 2024-09-15T01:33:05.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272182+0000", 2024-09-15T01:33:05.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2792e-05 2024-09-15T01:33:05.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272209+0000", 2024-09-15T01:33:05.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6409000000000001e-05 2024-09-15T01:33:05.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272435+0000", 2024-09-15T01:33:05.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000226691 2024-09-15T01:33:05.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295435+0000", 2024-09-15T01:33:05.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022999636 2024-09-15T01:33:05.982 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.982 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.982 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.982 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=34, reqid=client.4477.0:296, at_version=44'25, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:05.982 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:33:05.982 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.575857803000002, 2024-09-15T01:33:05.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024251137999999998, 2024-09-15T01:33:05.983 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.983 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.983 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:33:05.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271466+0000", 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8899999999999998e-07 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271488+0000", 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1875999999999999e-05 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271522+0000", 2024-09-15T01:33:05.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4041999999999997e-05 2024-09-15T01:33:05.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271529+0000", 2024-09-15T01:33:05.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4620000000000001e-06 2024-09-15T01:33:05.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272189+0000", 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066068699999999997 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272198+0000", 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2900000000000002e-06 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272284+0000", 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6638999999999995e-05 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272322+0000", 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7125000000000001e-05 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272357+0000", 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5626000000000001e-05 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272458+0000", 2024-09-15T01:33:05.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000100343 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295717+0000", 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023259058999999999 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=35, reqid=client.4477.0:291, at_version=0'0, trim_to=0'0, roll_forward_to=44'25))", 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:05.988 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.574250666000001, 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022393413000000001, 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:05.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:05.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0999999999999998e-07 2024-09-15T01:33:05.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273083+0000", 2024-09-15T01:33:05.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0015e-05 2024-09-15T01:33:05.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273110+0000", 2024-09-15T01:33:05.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6798999999999999e-05 2024-09-15T01:33:05.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273117+0000", 2024-09-15T01:33:05.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5379999999999999e-06 2024-09-15T01:33:05.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273155+0000", 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8563999999999999e-05 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273163+0000", 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4939999999999997e-06 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273248+0000", 2024-09-15T01:33:05.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5290000000000005e-05 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273254+0000", 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3149999999999997e-06 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273288+0000", 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3958000000000003e-05 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:05.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273303+0000", 2024-09-15T01:33:05.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4731e-05 2024-09-15T01:33:05.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:05.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295466+0000", 2024-09-15T01:33:05.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022162998999999999 2024-09-15T01:33:05.994 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.994 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:05.994 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:05.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.995 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 54/31 ECSubWrite(tid=63, reqid=client.4477.0:459, at_version=54'49, trim_to=0'0, roll_forward_to=54'46))", 2024-09-15T01:33:05.995 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.094073+0000", 2024-09-15T01:33:05.995 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 9.7532497760000005, 2024-09-15T01:33:05.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.82711023299999997, 2024-09-15T01:33:05.995 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:05.995 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:05.995 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:05.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:05.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094073+0000", 2024-09-15T01:33:05.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:05.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094073+0000", 2024-09-15T01:33:05.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:05.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:05.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094074+0000", 2024-09-15T01:33:05.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4099999999999999e-07 2024-09-15T01:33:05.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094105+0000", 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1087000000000002e-05 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920807+0000", 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.82670179300000002 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:05.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920813+0000", 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8100000000000003e-06 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920878+0000", 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4870999999999996e-05 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920885+0000", 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3860000000000001e-06 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920975+0000", 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9797999999999996e-05 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920981+0000", 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2419999999999997e-06 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.921033+0000", 2024-09-15T01:33:05.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1811999999999998e-05 2024-09-15T01:33:06.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.921048+0000", 2024-09-15T01:33:06.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5135e-05 2024-09-15T01:33:06.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.921184+0000", 2024-09-15T01:33:06.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013575800000000001 2024-09-15T01:33:06.000 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.000 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.001 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.001 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.001 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.001 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:06.001 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:33:06.002 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:06.002 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:06.002 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:06.003 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:06.003 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:06.003 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:06.004 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:06.004 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:05.933+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:06.005 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:06.005 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:06.005 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:06.006 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:06.006 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:06.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:05.989+0000 7f8aa7fff640 0 -- 172.21.15.42:0/2425046349 >> [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] conn(0x7f8aa8195d00 msgr2=0x7f8aa81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:06.049 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:06.047+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:06.071 INFO:tasks.rados.rados.0.smithi175.stdout:226: done (3 left) 2024-09-15T01:33:06.071 INFO:tasks.rados.rados.0.smithi175.stdout:227: done (2 left) 2024-09-15T01:33:06.071 INFO:tasks.rados.rados.0.smithi175.stdout:228: done (1 left) 2024-09-15T01:33:06.071 INFO:tasks.rados.rados.0.smithi175.stdout:230: done (0 left) 2024-09-15T01:33:06.071 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 23 current snap is 17 2024-09-15T01:33:06.072 INFO:tasks.rados.rados.0.smithi175.stdout:231: seq_num 114 ranges {376832=753664,1130496=688128,1818624=475136} 2024-09-15T01:33:06.077 INFO:tasks.rados.rados.0.smithi175.stdout:231: writing smithi17529310-23 from 376832 to 1130496 tid 1 2024-09-15T01:33:06.081 INFO:tasks.rados.rados.0.smithi175.stdout:231: writing smithi17529310-23 from 1130496 to 1818624 tid 2 2024-09-15T01:33:06.083 INFO:tasks.rados.rados.0.smithi175.stdout:231: writing smithi17529310-23 from 1818624 to 2293760 tid 3 2024-09-15T01:33:06.084 INFO:tasks.rados.rados.0.smithi175.stdout:232: rollback oid 34 current snap is 17 2024-09-15T01:33:06.084 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 34 to 13 2024-09-15T01:33:06.084 INFO:tasks.rados.rados.0.smithi175.stdout:233: snap_create 2024-09-15T01:33:06.086 INFO:tasks.rados.rados.0.smithi175.stdout:232: finishing rollback tid 1 to smithi17529310-34 2024-09-15T01:33:06.087 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 34 v 71 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T01:33:06.090 INFO:tasks.rados.rados.0.smithi175.stdout:231: finishing write tid 1 to smithi17529310-23 2024-09-15T01:33:06.090 INFO:tasks.rados.rados.0.smithi175.stdout:231: finishing write tid 2 to smithi17529310-23 2024-09-15T01:33:06.090 INFO:tasks.rados.rados.0.smithi175.stdout:231: finishing write tid 3 to smithi17529310-23 2024-09-15T01:33:06.090 INFO:tasks.rados.rados.0.smithi175.stdout:231: finishing write tid 4 to smithi17529310-23 2024-09-15T01:33:06.092 INFO:tasks.rados.rados.0.smithi175.stdout:231: finishing write tid 5 to smithi17529310-23 2024-09-15T01:33:06.092 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 23 v 38 (ObjNum 114 snap 17 seq_num 114) dirty exists 2024-09-15T01:33:06.092 INFO:tasks.rados.rados.0.smithi175.stdout:231: left oid 23 (ObjNum 114 snap 17 seq_num 114) 2024-09-15T01:33:06.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:33:06.105 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:06.106 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:06.106 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:06.106 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:06.114 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:33:06.136 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:06.134+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:06.211 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:06.211 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:06.237 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:06.236+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:06.252 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:06.252 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:06.252 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:06.252 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:06.252 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:06.260 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:33:06.338 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:06.337+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:06.359 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:06.359 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:06.359 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:06.359 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:06.359 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.359 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:06.359 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:06.359 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.115538566000001, 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022664864999999999, 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:06.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.16e-07 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241155+0000", 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016715400000000001 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241334+0000", 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017883600000000001 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241342+0000", 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1519999999999992e-06 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241446+0000", 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010419 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.362 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241468+0000", 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1747e-05 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.244959+0000", 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0034911600000000001 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.244970+0000", 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1131000000000001e-05 2024-09-15T01:33:06.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.364 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245013+0000", 2024-09-15T01:33:06.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3214000000000003e-05 2024-09-15T01:33:06.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.364 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245498+0000", 2024-09-15T01:33:06.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00048501499999999999 2024-09-15T01:33:06.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263652+0000", 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018154050000000001 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.115340606, 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022656372000000001, 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.365 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:06.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9399999999999999e-07 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241203+0000", 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7152000000000001e-05 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241707+0000", 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050421299999999999 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241714+0000", 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4409999999999998e-06 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241773+0000", 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9762000000000001e-05 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241783+0000", 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6860000000000001e-06 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245035+0000", 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032521130000000001 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245184+0000", 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014925800000000001 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245769+0000", 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058491399999999996 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246140+0000", 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037060199999999999 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263842+0000", 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.017702037 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.370 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.114123488000001, 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034719353000000001, 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1600000000000001e-07 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242427+0000", 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4022000000000001e-05 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242460+0000", 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2496000000000002e-05 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242476+0000", 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6503000000000001e-05 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245083+0000", 2024-09-15T01:33:06.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.002606531 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245092+0000", 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7679999999999997e-06 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245298+0000", 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000205971 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245772+0000", 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00047452200000000001 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245817+0000", 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4474999999999997e-05 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246171+0000", 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035433200000000001 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277122+0000", 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030951117 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.113499956999998, 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031640535999999997, 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.49e-07 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243044+0000", 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7946999999999999e-05 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243140+0000", 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5311999999999998e-05 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243147+0000", 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2069999999999998e-06 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245010+0000", 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018629230000000001 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245020+0000", 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0774999999999999e-05 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245472+0000", 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045200900000000002 2024-09-15T01:33:06.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245773+0000", 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00030029999999999998 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245817+0000", 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4367999999999999e-05 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246317+0000", 2024-09-15T01:33:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050021500000000003 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274667+0000", 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028349330999999998 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.109235910999999, 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027771575999999999, 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247291+0000", 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7700000000000004e-07 2024-09-15T01:33:06.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247302+0000", 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1246e-05 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247333+0000", 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1179000000000001e-05 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247341+0000", 2024-09-15T01:33:06.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.571e-06 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247388+0000", 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6876999999999999e-05 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247395+0000", 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3359999999999997e-06 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247478+0000", 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2860000000000003e-05 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247487+0000", 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9029999999999993e-06 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247526+0000", 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8829999999999999e-05 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.385 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247546+0000", 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0551999999999999e-05 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275062+0000", 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027515444999999999 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as5 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:06.386 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.109013091000001, 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027915809999999999, 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247514+0000", 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9899999999999999e-07 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247524+0000", 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0396e-05 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247555+0000", 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0414000000000001e-05 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247564+0000", 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1900000000000001e-06 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247616+0000", 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1788e-05 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247628+0000", 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.288e-05 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247763+0000", 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013473700000000001 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247772+0000", 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3429999999999996e-06 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247804+0000", 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2533999999999997e-05 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247817+0000", 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2694000000000001e-05 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275429+0000", 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027612035 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s5 44/31 ECSubWrite(tid=65, reqid=osd.14.0:65, at_version=44'26, trim_to=0'0, roll_forward_to=44'24))", 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:06.391 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.106967744999999, 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.026123550999999998, 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:06.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249559+0000", 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3200000000000003e-07 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249584+0000", 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4284999999999999e-05 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249633+0000", 2024-09-15T01:33:06.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9526e-05 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249641+0000", 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2239999999999995e-06 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249695+0000", 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3674999999999997e-05 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249705+0000", 2024-09-15T01:33:06.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8300000000000008e-06 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251518+0000", 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018128409999999999 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251549+0000", 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0948999999999998e-05 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251591+0000", 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2515999999999997e-05 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251612+0000", 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0494000000000001e-05 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275682+0000", 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024070279 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.396 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s5 44/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=44'26))", 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.103774035000001, 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022959355000000001, 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:06.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252753+0000", 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7499999999999998e-07 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252764+0000", 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1123e-05 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252899+0000", 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013444100000000001 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252906+0000", 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8650000000000001e-06 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252959+0000", 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2698000000000003e-05 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252968+0000", 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2140000000000002e-06 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253058+0000", 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9926999999999995e-05 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253066+0000", 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6620000000000005e-06 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253105+0000", 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8875999999999998e-05 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253117+0000", 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2405000000000001e-05 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275711+0000", 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022594169000000001 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=60, reqid=client.4477.0:306, at_version=44'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.085697634999999, 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024129333999999999, 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270829+0000", 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270845+0000", 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6013999999999999e-05 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270871+0000", 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6140999999999999e-05 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270877+0000", 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7989999999999999e-06 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270947+0000", 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9480000000000006e-05 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270959+0000", 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2395000000000001e-05 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274484+0000", 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003524976 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274715+0000", 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000230595 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274806+0000", 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1130000000000003e-05 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274847+0000", 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1473000000000001e-05 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.294958+0000", 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020110711 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.085619625, 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020359914999999999, 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270907+0000", 2024-09-15T01:33:06.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4200000000000001e-07 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270919+0000", 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1761e-05 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270936+0000", 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7761000000000002e-05 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270953+0000", 2024-09-15T01:33:06.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6858e-05 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272042+0000", 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010882769999999999 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272051+0000", 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8400000000000007e-06 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272145+0000", 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3639000000000003e-05 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272153+0000", 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6240000000000002e-06 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272221+0000", 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8281999999999994e-05 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272241+0000", 2024-09-15T01:33:06.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9542e-05 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.291266+0000", 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019025889000000001 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.084108379, 2024-09-15T01:33:06.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.025858174000000001, 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272446+0000", 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7114000000000002e-05 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272489+0000", 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3711999999999998e-05 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272498+0000", 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9169999999999995e-06 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274537+0000", 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0020384190000000001 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274545+0000", 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1999999999999994e-06 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274659+0000", 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011420499999999999 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274716+0000", 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7114999999999998e-05 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274807+0000", 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0346999999999994e-05 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274883+0000", 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.695e-05 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298276+0000", 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023392452000000001 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=62, reqid=client.4477.0:308, at_version=44'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:06.417 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.083061228999998, 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024943603000000002, 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.37e-07 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273478+0000", 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2768e-05 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273502+0000", 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3920000000000001e-05 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273508+0000", 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0909999999999998e-06 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274699+0000", 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011909240000000001 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274707+0000", 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5429999999999996e-06 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274788+0000", 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1804000000000005e-05 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274811+0000", 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2379e-05 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274902+0000", 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1031000000000003e-05 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274910+0000", 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8359999999999992e-06 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298408+0000", 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023498769999999999 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=92, reqid=client.4477.0:309, at_version=44'31, trim_to=0'0, roll_forward_to=44'28))", 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:06.422 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.081961751000001, 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024021724000000001, 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274565+0000", 2024-09-15T01:33:06.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9299999999999996e-07 2024-09-15T01:33:06.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275016+0000", 2024-09-15T01:33:06.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045075999999999997 2024-09-15T01:33:06.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275077+0000", 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0582999999999999e-05 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275084+0000", 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1289999999999998e-06 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275144+0000", 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0566000000000003e-05 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275153+0000", 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4849999999999992e-06 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275302+0000", 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000149689 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275327+0000", 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4337000000000001e-05 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277020+0000", 2024-09-15T01:33:06.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001693153 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277074+0000", 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4122999999999997e-05 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298586+0000", 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021512006 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.427 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 44/31 ECSubWrite(tid=35, reqid=client.4477.0:296, at_version=0'0, trim_to=0'0, roll_forward_to=44'25))", 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.078419777000001, 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020797596000000002, 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278107+0000", 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8199999999999998e-07 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278119+0000", 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1567e-05 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278155+0000", 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6560999999999998e-05 2024-09-15T01:33:06.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278163+0000", 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2819999999999999e-06 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278209+0000", 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6146999999999997e-05 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278219+0000", 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0699999999999999e-05 2024-09-15T01:33:06.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278342+0000", 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000122061 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278350+0000", 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4460000000000001e-06 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280088+0000", 2024-09-15T01:33:06.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0017376309999999999 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280107+0000", 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.986e-05 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298904+0000", 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018796459000000001 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=93, reqid=client.4477.0:310, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.077711774000001, 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020226037999999998, 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278815+0000", 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6000000000000003e-07 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279204+0000", 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000388413 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279228+0000", 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4674000000000001e-05 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279233+0000", 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7849999999999999e-06 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279272+0000", 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9502999999999998e-05 2024-09-15T01:33:06.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279278+0000", 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0079999999999999e-06 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279401+0000", 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012238799999999999 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280091+0000", 2024-09-15T01:33:06.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00069018900000000002 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280213+0000", 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012227700000000001 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280231+0000", 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7408999999999999e-05 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.299040+0000", 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018809632 2024-09-15T01:33:06.437 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.5s6 44/31 ECSubWrite(tid=15, reqid=client.4477.0:314, at_version=44'8, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.077162317999999, 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.026604415999999999, 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:06.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279386+0000", 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.211e-05 2024-09-15T01:33:06.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279421+0000", 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4523000000000001e-05 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279428+0000", 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4420000000000001e-06 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279474+0000", 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5995000000000003e-05 2024-09-15T01:33:06.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279480+0000", 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.105e-06 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279645+0000", 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016430199999999999 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280092+0000", 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044707999999999998 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280214+0000", 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012248399999999999 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280260+0000", 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5785000000000003e-05 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.305968+0000", 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.025708106000000001 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.5s6 44/31 ECSubWrite(tid=16, reqid=client.4477.0:314, at_version=0'0, trim_to=0'0, roll_forward_to=44'8))", 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.074507744000002, 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024148095000000001, 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282019+0000", 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282028+0000", 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0969999999999995e-06 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282054+0000", 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5897e-05 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282059+0000", 2024-09-15T01:33:06.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7849999999999999e-06 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282101+0000", 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1801000000000002e-05 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282108+0000", 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0650000000000001e-06 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282171+0000", 2024-09-15T01:33:06.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2834000000000002e-05 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282844+0000", 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00067343600000000002 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282915+0000", 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1242999999999995e-05 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282933+0000", 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8136000000000001e-05 2024-09-15T01:33:06.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306166+0000", 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023233052000000001 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=63, reqid=client.4477.0:308, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.072924111999999, 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022584495, 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.448 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283603+0000", 2024-09-15T01:33:06.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5899999999999995e-07 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283618+0000", 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5600999999999999e-05 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283648+0000", 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9295e-05 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283653+0000", 2024-09-15T01:33:06.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1050000000000003e-06 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283703+0000", 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9883999999999999e-05 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283712+0000", 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3740000000000004e-06 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283859+0000", 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000146959 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283869+0000", 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0295999999999999e-05 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283906+0000", 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6534e-05 2024-09-15T01:33:06.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283922+0000", 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6560000000000001e-05 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306187+0000", 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022264128000000001 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.454 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 47/38 ECSubWrite(tid=76, reqid=client.4477.0:376, at_version=47'25, trim_to=0'0, roll_forward_to=47'22))", 2024-09-15T01:33:06.454 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:06.454 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.992719949000001, 2024-09-15T01:33:06.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.61852187999999997, 2024-09-15T01:33:06.454 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.454 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.454 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:06.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363807+0000", 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5799999999999999e-07 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363818+0000", 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1729000000000001e-05 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981818+0000", 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.61799953399999996 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981826+0000", 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1049999999999995e-06 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981893+0000", 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7229000000000003e-05 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981905+0000", 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1527e-05 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982064+0000", 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000159147 2024-09-15T01:33:06.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982078+0000", 2024-09-15T01:33:06.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:33:06.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982122+0000", 2024-09-15T01:33:06.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4354000000000003e-05 2024-09-15T01:33:06.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982140+0000", 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8315e-05 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982328+0000", 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018781200000000001 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 54/52 ECSubWrite(tid=119, reqid=client.4477.0:500, at_version=54'41, trim_to=0'0, roll_forward_to=54'38))", 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 10.145224918, 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.27782579699999999, 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211302+0000", 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.89e-07 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211313+0000", 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1521e-05 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488807+0000", 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.27749379200000002 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488811+0000", 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2889999999999999e-06 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488867+0000", 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5322000000000003e-05 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488874+0000", 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9659999999999996e-06 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488953+0000", 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8591999999999996e-05 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488959+0000", 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0000000000000002e-06 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488993+0000", 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4326999999999998e-05 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.489009+0000", 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6015000000000002e-05 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.489127+0000", 2024-09-15T01:33:06.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117584 2024-09-15T01:33:06.465 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.465 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.465 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.465 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.465 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.465 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:06.466 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:33:06.467 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:06.464+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:06.564 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:06.564 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:06.565 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:06.565 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:06.567 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:06.565+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:06.572 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:33:06.668 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:06.666+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:06.669 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:06.669 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:06.669 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:06.670 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:06.670 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:06.678 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:33:06.769 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:06.767+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:06.818 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as6 39/38 ECSubWrite(tid=14, reqid=client.4477.0:213, at_version=39'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:41.207762+0000", 2024-09-15T01:33:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 25.607850130999999, 2024-09-15T01:33:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0079059590000000006, 2024-09-15T01:33:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.207762+0000", 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.207762+0000", 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.207763+0000", 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4000000000000002e-07 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208247+0000", 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00048424399999999999 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208277+0000", 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0545e-05 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208284+0000", 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.472e-06 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208339+0000", 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5250000000000001e-05 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.208347+0000", 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1829999999999994e-06 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.210632+0000", 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022842470000000001 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.210672+0000", 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0425e-05 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.210764+0000", 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2077000000000006e-05 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.211058+0000", 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00029391200000000001 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:41.215668+0000", 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046100639999999997 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 44/31 ECSubWrite(tid=82, reqid=client.4477.0:281, at_version=44'36, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.243189+0000", 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.572423292, 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0097687209999999993, 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243189+0000", 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243189+0000", 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243190+0000", 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1800000000000004e-07 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243219+0000", 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8903000000000001e-05 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243259+0000", 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0348000000000001e-05 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243267+0000", 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2490000000000001e-06 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243347+0000", 2024-09-15T01:33:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9412000000000005e-05 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243358+0000", 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1447999999999999e-05 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243828+0000", 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00046974499999999998 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243876+0000", 2024-09-15T01:33:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7651999999999997e-05 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245401+0000", 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015253230000000001 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245443+0000", 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2135999999999998e-05 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252958+0000", 2024-09-15T01:33:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0075145869999999997 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 44/31 ECSubWrite(tid=83, reqid=osd.11.0:83, at_version=44'38, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.245451+0000", 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.570161617, 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0089667800000000006, 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245451+0000", 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245451+0000", 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245451+0000", 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1399999999999996e-07 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245473+0000", 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2096e-05 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245526+0000", 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2858000000000001e-05 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245537+0000", 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.102e-05 2024-09-15T01:33:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245586+0000", 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8962999999999999e-05 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245596+0000", 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4970000000000007e-06 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245826+0000", 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022987600000000001 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245835+0000", 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0929999999999992e-06 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245874+0000", 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9480000000000001e-05 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245889+0000", 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4698000000000001e-05 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254417+0000", 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0085284850000000006 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 44/31 ECSubWrite(tid=36, reqid=osd.7.0:36, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.245936+0000", 2024-09-15T01:33:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.569675750000002, 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0086810180000000004, 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245936+0000", 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245936+0000", 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245937+0000", 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.91e-07 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245959+0000", 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2237000000000001e-05 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246056+0000", 2024-09-15T01:33:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6699000000000004e-05 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246063+0000", 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3409999999999997e-06 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246125+0000", 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1987000000000001e-05 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246135+0000", 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6819999999999998e-06 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246506+0000", 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037097200000000001 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246515+0000", 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0599999999999997e-06 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248321+0000", 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018059180000000001 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248338+0000", 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7699000000000001e-05 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254618+0000", 2024-09-15T01:33:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.006279232 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.568719413, 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011534543, 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246894+0000", 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0500000000000002e-07 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2585000000000001e-05 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246951+0000", 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4552999999999999e-05 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246961+0000", 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0516000000000001e-05 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247025+0000", 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3540999999999994e-05 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247039+0000", 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4484000000000001e-05 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247338+0000", 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00029853699999999998 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248324+0000", 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0009866059999999999 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248375+0000", 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0337000000000003e-05 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248387+0000", 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2081e-05 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258427+0000", 2024-09-15T01:33:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010040498 2024-09-15T01:33:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247016+0000", 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.568596499999998, 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0091827060000000005, 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247016+0000", 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247016+0000", 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247017+0000", 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9599999999999998e-07 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247044+0000", 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6995000000000002e-05 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247087+0000", 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3587999999999997e-05 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247093+0000", 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7810000000000002e-06 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247148+0000", 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5413e-05 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247158+0000", 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8910000000000007e-06 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247380+0000", 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000221993 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248325+0000", 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00094500499999999996 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248376+0000", 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0373e-05 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248394+0000", 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8666999999999999e-05 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256198+0000", 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078041040000000001 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as6 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.568075274000002, 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010349413, 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247538+0000", 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0130000000000001e-06 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247552+0000", 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3723999999999999e-05 2024-09-15T01:33:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247588+0000", 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5989999999999999e-05 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247595+0000", 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5789999999999998e-06 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247648+0000", 2024-09-15T01:33:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2757000000000003e-05 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247654+0000", 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3629999999999999e-06 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247737+0000", 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2886000000000006e-05 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248326+0000", 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058858900000000004 2024-09-15T01:33:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248380+0000", 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3832999999999998e-05 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248401+0000", 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0843e-05 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257886+0000", 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0094858359999999992 2024-09-15T01:33:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.6s6 44/38 ECSubWrite(tid=21, reqid=osd.0.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.248975+0000", 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.566637263000001, 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0092777850000000002, 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248975+0000", 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248975+0000", 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248975+0000", 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1199999999999999e-07 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248984+0000", 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9020000000000005e-06 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249241+0000", 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000256331 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249247+0000", 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9489999999999999e-06 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249305+0000", 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7503000000000002e-05 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249315+0000", 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0253e-05 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249436+0000", 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120547 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251159+0000", 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001723694 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251211+0000", 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1356000000000001e-05 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251238+0000", 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6874000000000001e-05 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258253+0000", 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0070150639999999997 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.556543919999999, 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015398044, 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259069+0000", 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2399999999999996e-07 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259095+0000", 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5527000000000001e-05 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259133+0000", 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8211999999999999e-05 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259140+0000", 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9330000000000004e-06 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259189+0000", 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8797000000000001e-05 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259200+0000", 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0932e-05 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259425+0000", 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000225505 2024-09-15T01:33:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259453+0000", 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8152000000000001e-05 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.261606+0000", 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021525870000000001 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.261621+0000", 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4829e-05 2024-09-15T01:33:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274466+0000", 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012845646 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 44/31 ECSubWrite(tid=39, reqid=client.4477.0:283, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.547219711, 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014946894, 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0299999999999998e-07 2024-09-15T01:33:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268593+0000", 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000199278 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268633+0000", 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0367000000000002e-05 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268640+0000", 2024-09-15T01:33:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5100000000000001e-06 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268742+0000", 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010184899999999999 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268752+0000", 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9299999999999998e-06 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269007+0000", 2024-09-15T01:33:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000255016 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269035+0000", 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8039999999999999e-05 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271592+0000", 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025566260000000002 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271610+0000", 2024-09-15T01:33:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7629999999999999e-05 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283339+0000", 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011729844999999999 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.6s6 44/38 ECSubWrite(tid=22, reqid=client.4477.0:293, at_version=44'14, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.544436189999999, 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012212746, 2024-09-15T01:33:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271177+0000", 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271394+0000", 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021719400000000001 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271421+0000", 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6842000000000002e-05 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271426+0000", 2024-09-15T01:33:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9010000000000002e-06 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271464+0000", 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7928e-05 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271476+0000", 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1979e-05 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271800+0000", 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032464100000000002 2024-09-15T01:33:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271832+0000", 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1501999999999998e-05 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271921+0000", 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9514999999999999e-05 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271940+0000", 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8436000000000001e-05 2024-09-15T01:33:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283389+0000", 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011449086000000001 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 44/31 ECSubWrite(tid=92, reqid=client.4477.0:309, at_version=44'31, trim_to=0'0, roll_forward_to=44'28))", 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.274881+0000", 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.540730838999998, 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0087926889999999994, 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274881+0000", 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274881+0000", 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274882+0000", 2024-09-15T01:33:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8899999999999998e-07 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275493+0000", 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061061299999999998 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275531+0000", 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8254999999999999e-05 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275542+0000", 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1228e-05 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275591+0000", 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8142000000000001e-05 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275602+0000", 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1274000000000001e-05 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275820+0000", 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021839299999999999 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275828+0000", 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6769999999999995e-06 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277082+0000", 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012540769999999999 2024-09-15T01:33:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277121+0000", 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8665999999999999e-05 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283674+0000", 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0065534749999999996 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 44/31 ECSubWrite(tid=43, reqid=client.4477.0:283, at_version=0'0, trim_to=0'0, roll_forward_to=44'32))", 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.275111+0000", 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.540501189, 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083635600000000008, 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275111+0000", 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275111+0000", 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275112+0000", 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4799999999999997e-07 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275129+0000", 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7022999999999999e-05 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275167+0000", 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7657000000000002e-05 2024-09-15T01:33:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275174+0000", 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7100000000000007e-06 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275220+0000", 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5266999999999999e-05 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275230+0000", 2024-09-15T01:33:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0169e-05 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275330+0000", 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000100673 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275341+0000", 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0451e-05 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275376+0000", 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4767000000000001e-05 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275391+0000", 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5119e-05 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283475+0000", 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0080837760000000008 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.9s6 44/31 ECSubWrite(tid=67, reqid=client.4477.0:315, at_version=44'20, trim_to=0'0, roll_forward_to=44'18))", 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.303974+0000", 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.511637824000001, 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0093094879999999994, 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.303974+0000", 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.303974+0000", 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.303976+0000", 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.916e-06 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.304088+0000", 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112069 2024-09-15T01:33:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.304159+0000", 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0110999999999993e-05 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.304172+0000", 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3668e-05 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.304220+0000", 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7849999999999998e-05 2024-09-15T01:33:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.304232+0000", 2024-09-15T01:33:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1929999999999999e-05 2024-09-15T01:33:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.304514+0000", 2024-09-15T01:33:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00028180799999999998 2024-09-15T01:33:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.304525+0000", 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1258e-05 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.305022+0000", 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000496866 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.305042+0000", 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9641999999999999e-05 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.313284+0000", 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082423700000000006 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 55/31 ECSubWrite(tid=160, reqid=client.4477.0:512, at_version=55'53, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 9.7399355159999992, 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011287287999999999, 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075698+0000", 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0384999999999999e-05 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075734+0000", 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6300999999999997e-05 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075743+0000", 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9549999999999998e-06 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075803+0000", 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0304999999999999e-05 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075813+0000", 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9539999999999999e-06 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075916+0000", 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102534 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075925+0000", 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1509999999999993e-06 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075971+0000", 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6276999999999998e-05 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075995+0000", 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3176999999999998e-05 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.086964+0000", 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010969504999999999 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 55/31 ECSubWrite(tid=161, reqid=client.4477.0:513, at_version=55'54, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 9.7392990719999997, 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010838802, 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076314+0000", 2024-09-15T01:33:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076329+0000", 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5505000000000001e-05 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076356+0000", 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6324000000000001e-05 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076362+0000", 2024-09-15T01:33:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2820000000000002e-06 2024-09-15T01:33:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076409+0000", 2024-09-15T01:33:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7190000000000001e-05 2024-09-15T01:33:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076419+0000", 2024-09-15T01:33:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9450000000000005e-06 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076491+0000", 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1680999999999994e-05 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076497+0000", 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3810000000000004e-06 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076958+0000", 2024-09-15T01:33:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00046042499999999998 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076971+0000", 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2999999999999999e-05 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.087152+0000", 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010181366000000001 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 55/31 ECSubWrite(tid=56, reqid=client.4477.0:518, at_version=55'46, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.076867+0000", 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 9.7387452769999996, 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010241029, 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076867+0000", 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076867+0000", 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076868+0000", 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9500000000000001e-07 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076882+0000", 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4357e-05 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076915+0000", 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2954999999999999e-05 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076920+0000", 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1270000000000002e-06 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076957+0000", 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6665000000000002e-05 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076966+0000", 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3139999999999993e-06 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.077038+0000", 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1323999999999997e-05 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.077045+0000", 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4839999999999998e-06 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.077372+0000", 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032736700000000001 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.077388+0000", 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5744999999999999e-05 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.087108+0000", 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0097197959999999993 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 55/31 ECSubWrite(tid=162, reqid=client.4477.0:513, at_version=0'0, trim_to=0'0, roll_forward_to=55'54))", 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.078563+0000", 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 9.7370491819999998, 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090906349999999997, 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.078563+0000", 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.078563+0000", 2024-09-15T01:33:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.078564+0000", 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.85e-07 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.078575+0000", 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1433999999999999e-05 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.078615+0000", 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9578e-05 2024-09-15T01:33:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.078622+0000", 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7800000000000003e-06 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.078669+0000", 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7552999999999997e-05 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.078679+0000", 2024-09-15T01:33:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0246e-05 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.078810+0000", 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000130541 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.078818+0000", 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6499999999999996e-06 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.079042+0000", 2024-09-15T01:33:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000224057 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.079067+0000", 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5775999999999999e-05 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.087654+0000", 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0085862349999999994 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 59/31 ECSubWrite(tid=179, reqid=client.4477.0:552, at_version=59'58, trim_to=0'0, roll_forward_to=56'56))", 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.7241298030000003, 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010784815, 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091483+0000", 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2999999999999999e-07 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091650+0000", 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016715 2024-09-15T01:33:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091673+0000", 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3434999999999998e-05 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091680+0000", 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7329999999999997e-06 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091721+0000", 2024-09-15T01:33:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1088999999999999e-05 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091729+0000", 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9170000000000006e-06 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091940+0000", 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021130000000000001 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.092580+0000", 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00063981200000000002 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093157+0000", 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057673800000000003 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093178+0000", 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0911e-05 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.102267+0000", 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090895000000000004 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 59/31 ECSubWrite(tid=182, reqid=client.4477.0:552, at_version=0'0, trim_to=0'0, roll_forward_to=59'58))", 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:01.093537+0000", 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.7220754549999997, 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0087873199999999995, 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093537+0000", 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093537+0000", 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093537+0000", 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1099999999999995e-07 2024-09-15T01:33:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093550+0000", 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2405999999999999e-05 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093575+0000", 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5225999999999999e-05 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093580+0000", 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5970000000000002e-06 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093626+0000", 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6177999999999997e-05 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093636+0000", 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0331e-05 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093751+0000", 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000115339 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093759+0000", 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1829999999999997e-06 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.096054+0000", 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022954759999999999 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.096071+0000", 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6477000000000002e-05 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.102324+0000", 2024-09-15T01:33:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0062534959999999999 2024-09-15T01:33:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:06.923 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:06.925 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-15T01:33:06.925 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:06.923+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:07.023 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:07.023 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:07.023 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:07.023 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:07.025 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:07.024+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:07.032 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-15T01:33:07.072 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-15T01:33:07.073 INFO:tasks.rados.rados.0.smithi175.stdout:231: done (2 left) 2024-09-15T01:33:07.073 INFO:tasks.rados.rados.0.smithi175.stdout:232: done (1 left) 2024-09-15T01:33:07.073 INFO:tasks.rados.rados.0.smithi175.stdout:233: done (0 left) 2024-09-15T01:33:07.073 INFO:tasks.rados.rados.0.smithi175.stdout:234: snap_remove snap 10 2024-09-15T01:33:07.126 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:07.124+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:07.169 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:07.169 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:07.169 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:07.170 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:07.170 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:07.177 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-15T01:33:07.227 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:07.225+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:07.276 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:07.276 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:74 3.0s0 3:0187269c:::smithi17529310-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 31.063455093999998, 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.071208814999999995, 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:07.277 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 74 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1200000000000003e-07 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210620+0000", 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.278e-06 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210623+0000", 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5779999999999999e-06 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210626+0000", 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.295e-06 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211745+0000", 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011191770000000001 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211880+0000", 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013497499999999999 2024-09-15T01:33:07.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212353+0000", 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000473616 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212441+0000", 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7268999999999994e-05 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212454+0000", 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3529999999999999e-05 2024-09-15T01:33:07.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212483+0000", 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9224e-05 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212755+0000", 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00027191599999999998 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.281770+0000", 2024-09-15T01:33:07.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.069014438999999997 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.281826+0000", 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6005999999999999e-05 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:75 3.0s0 3:0187269c:::smithi17529310-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:07.283 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 31.063422853999999, 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.072451382999999994, 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 75 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4700000000000002e-07 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210652+0000", 2024-09-15T01:33:07.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9160000000000001e-06 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210657+0000", 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0529999999999997e-06 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210660+0000", 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4339999999999999e-06 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212500+0000", 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001839732 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212584+0000", 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4759999999999995e-05 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.282036+0000", 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.069451322999999995 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.282131+0000", 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.577e-05 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.283100+0000", 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00096904800000000002 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds2 34/31 ECSubWrite(tid=16, reqid=client.4477.0:116, at_version=34'10, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 31.014448405, 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.24562283800000001, 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9599999999999999e-07 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259634+0000", 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0412e-05 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.504838+0000", 2024-09-15T01:33:07.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.245203848 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.504852+0000", 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3346999999999999e-05 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505000+0000", 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000148209 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505006+0000", 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6309999999999997e-06 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505068+0000", 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1962000000000001e-05 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505074+0000", 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0579999999999997e-06 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505103+0000", 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9828e-05 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505116+0000", 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2261e-05 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505246+0000", 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000130786 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.293 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds2 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 21.027155757999999, 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033387655000000002, 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:07.294 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246917+0000", 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.61e-07 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246937+0000", 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9741999999999998e-05 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246964+0000", 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7013999999999999e-05 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246970+0000", 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8250000000000001e-06 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247020+0000", 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0875999999999999e-05 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247032+0000", 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1280999999999999e-05 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249500+0000", 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0024682720000000001 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251106+0000", 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001606489 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251175+0000", 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8926999999999999e-05 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251258+0000", 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2441000000000006e-05 2024-09-15T01:33:07.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280304+0000", 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029046026999999999 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs2 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247251+0000", 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 21.026821131999998, 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033110460000000001, 2024-09-15T01:33:07.299 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247251+0000", 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247251+0000", 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247252+0000", 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9899999999999999e-07 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247274+0000", 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2160000000000001e-05 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247314+0000", 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0438000000000001e-05 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247321+0000", 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.187e-06 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249348+0000", 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.002026524 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249356+0000", 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7319999999999998e-06 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249538+0000", 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000182239 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251107+0000", 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001569094 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251176+0000", 2024-09-15T01:33:07.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8707999999999993e-05 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251269+0000", 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3246999999999997e-05 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280361+0000", 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029092331999999999 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.304 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s5 44/31 ECSubWrite(tid=28, reqid=osd.8.0:28, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.250215+0000", 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 21.023856962, 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033811606000000001, 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250215+0000", 2024-09-15T01:33:07.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250215+0000", 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250216+0000", 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3200000000000003e-07 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250240+0000", 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3849000000000002e-05 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250277+0000", 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7469999999999999e-05 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250284+0000", 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0979999999999996e-06 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250325+0000", 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0184e-05 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250331+0000", 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5060000000000001e-06 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254385+0000", 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0040539560000000001 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254397+0000", 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2502e-05 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254432+0000", 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4631000000000001e-05 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254987+0000", 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000555011 2024-09-15T01:33:07.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.284027+0000", 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029039466999999999 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s5 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.251962+0000", 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 21.022109647000001, 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034678124999999997, 2024-09-15T01:33:07.310 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251962+0000", 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251962+0000", 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251963+0000", 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4799999999999998e-07 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251986+0000", 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3221e-05 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252099+0000", 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112687 2024-09-15T01:33:07.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252107+0000", 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0399999999999993e-06 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254434+0000", 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0023271300000000002 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254447+0000", 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2607e-05 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254678+0000", 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000230869 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254847+0000", 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016962300000000001 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254891+0000", 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3441e-05 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.255014+0000", 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012353200000000001 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286640+0000", 2024-09-15T01:33:07.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031626327000000003 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs2 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.259073+0000", 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 21.014998862999999, 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031659406000000001, 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259073+0000", 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259073+0000", 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259074+0000", 2024-09-15T01:33:07.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1399999999999997e-07 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259092+0000", 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8604e-05 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259128+0000", 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6183999999999998e-05 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.318 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259136+0000", 2024-09-15T01:33:07.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3359999999999997e-06 2024-09-15T01:33:07.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259179+0000", 2024-09-15T01:33:07.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3198000000000002e-05 2024-09-15T01:33:07.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259187+0000", 2024-09-15T01:33:07.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4339999999999992e-06 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259585+0000", 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000397196 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259607+0000", 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2558999999999999e-05 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259635+0000", 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7538e-05 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259643+0000", 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6330000000000002e-06 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.290732+0000", 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.03108911 2024-09-15T01:33:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs2 44/31 ECSubWrite(tid=91, reqid=osd.11.0:87, at_version=0'0, trim_to=0'0, roll_forward_to=44'44))", 2024-09-15T01:33:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267434+0000", 2024-09-15T01:33:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 21.006637987000001, 2024-09-15T01:33:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039301718999999999, 2024-09-15T01:33:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267434+0000", 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267434+0000", 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267435+0000", 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.92e-07 2024-09-15T01:33:07.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267445+0000", 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8949999999999993e-06 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267477+0000", 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2768000000000002e-05 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.324 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267485+0000", 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4499999999999998e-06 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281136+0000", 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013651389999999999 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281144+0000", 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9549999999999992e-06 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281264+0000", 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012005799999999999 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282867+0000", 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00160271 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282917+0000", 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9919000000000001e-05 2024-09-15T01:33:07.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282948+0000", 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1053000000000002e-05 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306736+0000", 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023787729000000001 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s6 44/38 ECSubWrite(tid=60, reqid=client.4477.0:306, at_version=44'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270872+0000", 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 21.003199925000001, 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039161770999999998, 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270872+0000", 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.328 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270872+0000", 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270873+0000", 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0219999999999999e-06 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270902+0000", 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9079000000000001e-05 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.329 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270938+0000", 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5849999999999997e-05 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270958+0000", 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9762000000000002e-05 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280987+0000", 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010029733000000001 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280996+0000", 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0280000000000006e-06 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281214+0000", 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021798100000000001 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282866+0000", 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001651292 2024-09-15T01:33:07.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282916+0000", 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0145000000000002e-05 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282941+0000", 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5066e-05 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.310034+0000", 2024-09-15T01:33:07.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027092813 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s4 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270969+0000", 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 21.003102756000001, 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034685404000000003, 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.333 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270969+0000", 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270969+0000", 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270970+0000", 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6199999999999998e-07 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.334 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270986+0000", 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6622000000000001e-05 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271019+0000", 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3077000000000002e-05 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271025+0000", 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8699999999999997e-06 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.335 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274545+0000", 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0035193350000000002 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274553+0000", 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2810000000000008e-06 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274633+0000", 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0358000000000005e-05 2024-09-15T01:33:07.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274827+0000", 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019425000000000001 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.276977+0000", 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021494420000000001 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.276994+0000", 2024-09-15T01:33:07.337 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6733000000000001e-05 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.305655+0000", 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028660973999999999 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s4 44/31 ECSubWrite(tid=34, reqid=client.4477.0:296, at_version=44'25, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 21.002589443000002, 2024-09-15T01:33:07.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034642487, 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2600000000000002e-07 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271500+0000", 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6745999999999999e-05 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271570+0000", 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0242999999999998e-05 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.340 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271578+0000", 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0870000000000007e-06 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274674+0000", 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030959379999999999 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274682+0000", 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9570000000000002e-06 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.341 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274766+0000", 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3630999999999999e-05 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274828+0000", 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2609999999999999e-05 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.276978+0000", 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021496520000000002 2024-09-15T01:33:07.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277024+0000", 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5723999999999998e-05 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306125+0000", 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029101373 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s6 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.272445+0000", 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 21.001626626, 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041201330000000001, 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272445+0000", 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272445+0000", 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272446+0000", 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8299999999999995e-07 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272467+0000", 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0319999999999999e-05 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.345 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272505+0000", 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8862999999999997e-05 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272513+0000", 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.362e-06 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281243+0000", 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0087304259999999995 2024-09-15T01:33:07.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.347 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.347 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281250+0000", 2024-09-15T01:33:07.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9399999999999996e-06 2024-09-15T01:33:07.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.347 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.347 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281315+0000", 2024-09-15T01:33:07.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4664000000000003e-05 2024-09-15T01:33:07.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.347 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282868+0000", 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001552793 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282917+0000", 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9567999999999997e-05 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282975+0000", 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7617000000000001e-05 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.348 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.313647+0000", 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030671894000000002 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s6 44/38 ECSubWrite(tid=62, reqid=client.4477.0:308, at_version=44'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273497+0000", 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 21.000574756999999, 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.040229292, 2024-09-15T01:33:07.349 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273497+0000", 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273497+0000", 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.350 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273498+0000", 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2799999999999996e-07 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273514+0000", 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5942000000000001e-05 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273538+0000", 2024-09-15T01:33:07.351 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4705000000000001e-05 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273543+0000", 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9649999999999999e-06 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281338+0000", 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0077948050000000001 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.352 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281343+0000", 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6009999999999997e-06 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281391+0000", 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7768000000000001e-05 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282868+0000", 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001477587 2024-09-15T01:33:07.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282917+0000", 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9168000000000001e-05 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282987+0000", 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9294000000000004e-05 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.313726+0000", 2024-09-15T01:33:07.354 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030739829 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=100, last_complete=54'36, committed=1, applied=1))", 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 11.177663086000001, 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.063605834999999999, 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.355 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.01e-07 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.356 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096412+0000", 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.836e-06 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.159953+0000", 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.063540907999999993 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.159956+0000", 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0489999999999999e-06 2024-09-15T01:33:07.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.159987+0000", 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0725999999999997e-05 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.160015+0000", 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7915e-05 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.358 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:485 3.9s0 3:98b2f2fd:::smithi17529310-14:head [setxattr _header (57) in=7b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 11.109699395, 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40475048200000002, 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 485 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.359 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6199999999999998e-07 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164375+0000", 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0650000000000001e-06 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164380+0000", 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9810000000000003e-06 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164383+0000", 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.241e-06 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165443+0000", 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010601409999999999 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165554+0000", 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011091600000000001 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165888+0000", 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000334125 2024-09-15T01:33:07.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165960+0000", 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1601999999999996e-05 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165971+0000", 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0903999999999999e-05 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.363 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166005+0000", 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3525999999999997e-05 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166156+0000", 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000151077 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569047+0000", 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.402891102 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569123+0000", 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6340000000000004e-05 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:486 3.9s0 3:98b2f2fd:::smithi17529310-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 11.109673344000001, 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40586654100000003, 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:33:07.365 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 486 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:07.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2800000000000003e-07 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164402+0000", 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3440000000000001e-06 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164409+0000", 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6839999999999999e-06 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164417+0000", 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9140000000000008e-06 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166021+0000", 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016044189999999999 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.368 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166112+0000", 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0402000000000006e-05 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569421+0000", 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40330946099999998 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569534+0000", 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011237499999999999 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.570265+0000", 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00073161400000000001 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=106, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 11.107874267, 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.10267822, 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.370 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:07.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0399999999999996e-07 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166203+0000", 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2889999999999999e-06 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268821+0000", 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.102618771 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268827+0000", 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1140000000000002e-06 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268865+0000", 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7492e-05 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268876+0000", 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.095e-05 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.373 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=106, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.166241+0000", 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 11.107830485999999, 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.075762497999999998, 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166241+0000", 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166241+0000", 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166242+0000", 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6400000000000003e-07 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166244+0000", 2024-09-15T01:33:07.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3309999999999998e-06 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.241879+0000", 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.075634921999999993 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.241898+0000", 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9082000000000001e-05 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.241991+0000", 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2833999999999999e-05 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.242004+0000", 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2865e-05 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=106, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.166248+0000", 2024-09-15T01:33:07.377 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 11.107823934000001, 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40313786299999999, 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166248+0000", 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166248+0000", 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166249+0000", 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6700000000000003e-07 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166251+0000", 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.934e-06 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.568876+0000", 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.402625446 2024-09-15T01:33:07.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.568887+0000", 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0553e-05 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.568964+0000", 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7303000000000002e-05 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569386+0000", 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00042205999999999998 2024-09-15T01:33:07.380 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.381 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.381 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.381 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:07.381 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:07.381 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:07.381 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15] out_osds: [9] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-15T01:33:07.381 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:33:07.381 INFO:tasks.thrashosds.thrasher:Adding osd 9 2024-09-15T01:33:07.382 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 9 2024-09-15T01:33:07.383 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:07.380+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:07.384 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:33:07.483 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:07.481+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:07.493 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:07.494 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:07.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:33:07.584 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:07.582+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:07.639 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:07.639 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:07.639 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:07.639 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:07.639 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:07.648 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:33:07.682 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-15T01:33:07.680+0000 7f79afbf1640 -1 mon.a@0(leader).osd e65 definitely_dead 0 2024-09-15T01:33:07.685 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:07.683+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:07.748 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:07.748 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:07.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:07.748 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4302.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:23.147530+0000", 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 44.597155504, 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0086012669999999992, 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4302", 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.42:0/3794971532", 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 2 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.749 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147530+0000", 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147530+0000", 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147533+0000", 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.587e-06 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147545+0000", 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1912999999999999e-05 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147569+0000", 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4389e-05 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147583+0000", 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3913e-05 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147699+0000", 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116353 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.147705+0000", 2024-09-15T01:33:07.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7289999999999997e-06 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.155973+0000", 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0082681109999999999 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:23.156131+0000", 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015827199999999999 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.753 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:3 3.as0 3:5e9da7d3:::smithi17529310-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.174795+0000", 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.569890754999999, 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0075856939999999996, 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.174795+0000", 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.174795+0000", 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.174795+0000", 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3799999999999997e-07 2024-09-15T01:33:07.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.174797+0000", 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.818e-06 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.174801+0000", 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2089999999999999e-06 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.174804+0000", 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9720000000000001e-06 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176513+0000", 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001709052 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176585+0000", 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1571999999999998e-05 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.179928+0000", 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0033435130000000002 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.180018+0000", 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9316999999999999e-05 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.182381+0000", 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023627029999999999 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:192 3.as0 3:5e9da7d3:::smithi17529310-1:head [copy-from ver 6] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.552264859000001, 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01403071, 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 192 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.759 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192423+0000", 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3140000000000002e-06 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192431+0000", 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2320000000000001e-06 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192444+0000", 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3239e-05 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192454+0000", 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2439999999999999e-06 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192781+0000", 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032732699999999998 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192904+0000", 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012317900000000001 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.201873+0000", 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089691839999999998 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202377+0000", 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00050339000000000004 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202392+0000", 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5201999999999999e-05 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202869+0000", 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047669800000000001 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202888+0000", 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9202e-05 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203431+0000", 2024-09-15T01:33:07.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00054287499999999995 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.206451+0000", 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003020624 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:193 3.as0 3:5e9da7d3:::smithi17529310-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192463+0000", 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.552222425, 2024-09-15T01:33:07.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010399754000000001, 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 193 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192463+0000", 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192463+0000", 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192464+0000", 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192466+0000", 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5390000000000002e-06 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192476+0000", 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6549999999999999e-06 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192483+0000", 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9550000000000003e-06 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.193098+0000", 2024-09-15T01:33:07.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00061465200000000004 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.193155+0000", 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7608999999999997e-05 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202537+0000", 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093813579999999994 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:07.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202652+0000", 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115473 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202863+0000", 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000210826 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 37/31 ECSubWrite(tid=20, reqid=client.4477.0:187, at_version=37'13, trim_to=0'0, roll_forward_to=34'10))", 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.194379+0000", 2024-09-15T01:33:07.770 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.550306113000001, 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0086632730000000008, 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194379+0000", 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194379+0000", 2024-09-15T01:33:07.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194380+0000", 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7199999999999997e-07 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194404+0000", 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3587999999999998e-05 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194449+0000", 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4790999999999999e-05 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194457+0000", 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9640000000000003e-06 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194518+0000", 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1351000000000003e-05 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194528+0000", 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5999999999999996e-06 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194617+0000", 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9090000000000003e-05 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194627+0000", 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0029e-05 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194662+0000", 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5179000000000003e-05 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194679+0000", 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6816e-05 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203043+0000", 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0083638930000000007 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.775 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 37/31 ECSubWrite(tid=21, reqid=client.4477.0:188, at_version=37'14, trim_to=0'0, roll_forward_to=34'10))", 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.195155+0000", 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.549530347000001, 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0080205039999999995, 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195155+0000", 2024-09-15T01:33:07.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195155+0000", 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195156+0000", 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8299999999999996e-07 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195177+0000", 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1359999999999999e-05 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195213+0000", 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5194000000000002e-05 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195226+0000", 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3623e-05 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195265+0000", 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8532999999999999e-05 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195274+0000", 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2539999999999998e-06 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195386+0000", 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111734 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195397+0000", 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1589e-05 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.197123+0000", 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017253240000000001 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.197142+0000", 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9366000000000001e-05 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203176+0000", 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0060337439999999997 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:214 3.as0 3:5d5028c6:::smithi17529310-40:head [append 0~376832] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:41.208569+0000", 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.536116832000001, 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0082088490000000007, 2024-09-15T01:33:07.781 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 214 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208569+0000", 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208569+0000", 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208571+0000", 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1059999999999998e-06 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208873+0000", 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000301988 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208889+0000", 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6033999999999999e-05 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208896+0000", 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5630000000000003e-06 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208957+0000", 2024-09-15T01:33:07.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0102999999999997e-05 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.209065+0000", 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108523 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.209750+0000", 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00068517899999999995 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.209839+0000", 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8289999999999997e-05 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210121+0000", 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028203499999999998 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210219+0000", 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8450999999999996e-05 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210793+0000", 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00057359000000000002 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.216566+0000", 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005773172 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.216778+0000", 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021181499999999999 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.787 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:215 3.as0 3:5d5028c6:::smithi17529310-40:head [setxattr _header (56) in=7b] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:41.208914+0000", 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.535771951000001, 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0081133950000000007, 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 215 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.788 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208914+0000", 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208914+0000", 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208915+0000", 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8999999999999995e-07 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208919+0000", 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2529999999999998e-06 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208926+0000", 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1250000000000004e-06 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208929+0000", 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1549999999999999e-06 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.209920+0000", 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00099067899999999995 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.209999+0000", 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8782999999999994e-05 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210390+0000", 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039110000000000002 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210491+0000", 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010117599999999999 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210501+0000", 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0295000000000001e-05 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210677+0000", 2024-09-15T01:33:07.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017592700000000001 2024-09-15T01:33:07.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210794+0000", 2024-09-15T01:33:07.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011722099999999999 2024-09-15T01:33:07.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:07.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.216927+0000", 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0061331770000000001 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.217027+0000", 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9613999999999999e-05 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:216 3.as0 3:5d5028c6:::smithi17529310-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:41.208948+0000", 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.535737815000001, 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0098072100000000002, 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 216 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208948+0000", 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208948+0000", 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208948+0000", 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999997e-07 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208950+0000", 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1710000000000001e-06 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208958+0000", 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6599999999999995e-06 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208963+0000", 2024-09-15T01:33:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4449999999999998e-06 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210679+0000", 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017165189999999999 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210762+0000", 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2719000000000005e-05 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.217447+0000", 2024-09-15T01:33:07.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0066855090000000001 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.217557+0000", 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011001 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.218755+0000", 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011976630000000001 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 44/31 ECSubWrite(tid=36, reqid=osd.7.0:36, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.245895+0000", 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.498790884999998, 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0067445059999999999, 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245895+0000", 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245895+0000", 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245896+0000", 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.125e-06 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245943+0000", 2024-09-15T01:33:07.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7172000000000002e-05 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246011+0000", 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8176999999999995e-05 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246020+0000", 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0559999999999994e-06 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246086+0000", 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6048000000000002e-05 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246097+0000", 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0654e-05 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246955+0000", 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085852599999999997 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247002+0000", 2024-09-15T01:33:07.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6229000000000001e-05 2024-09-15T01:33:07.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247047+0000", 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5597999999999999e-05 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247063+0000", 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5675000000000002e-05 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252639+0000", 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005576246 2024-09-15T01:33:07.806 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.497710648000002, 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013669871, 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246976+0000", 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9800000000000002e-07 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247000+0000", 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4314000000000001e-05 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247045+0000", 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4938000000000002e-05 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247053+0000", 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:33:07.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247132+0000", 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8799000000000002e-05 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247142+0000", 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0511e-05 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247515+0000", 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037233299999999997 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247546+0000", 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1102000000000001e-05 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247585+0000", 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9152000000000001e-05 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247603+0000", 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.260645+0000", 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013041662000000001 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.812 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 44/31 ECSubWrite(tid=38, reqid=osd.7.0:37, at_version=0'0, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.251749+0000", 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.492936388, 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0092966969999999996, 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251749+0000", 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251749+0000", 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251750+0000", 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6899999999999996e-07 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251765+0000", 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.489e-05 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251796+0000", 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0972000000000001e-05 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251801+0000", 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3000000000000001e-06 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251845+0000", 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3900999999999999e-05 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251851+0000", 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3380000000000001e-06 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251966+0000", 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011479700000000001 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252286+0000", 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031985800000000001 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252343+0000", 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6909e-05 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252374+0000", 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0596000000000003e-05 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.261046+0000", 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0086723670000000003 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.817 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 48/31 ECSubWrite(tid=171, reqid=client.4477.0:402, at_version=48'24, trim_to=0'0, roll_forward_to=45'21))", 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:50.294480+0000", 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.450205932999999, 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0077438680000000001, 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294480+0000", 2024-09-15T01:33:07.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294480+0000", 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294480+0000", 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3099999999999997e-07 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294818+0000", 2024-09-15T01:33:07.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033747399999999999 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294863+0000", 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4987000000000001e-05 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294874+0000", 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1514e-05 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294927+0000", 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2629e-05 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.294937+0000", 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0382e-05 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295264+0000", 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032685400000000001 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295274+0000", 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5510000000000005e-06 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295383+0000", 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010977800000000001 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295406+0000", 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2908000000000002e-05 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.302224+0000", 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0068171600000000001 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 48/31 ECSubWrite(tid=172, reqid=client.4477.0:403, at_version=48'25, trim_to=0'0, roll_forward_to=45'21))", 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:50.295531+0000", 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.449154716999999, 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0068974090000000002, 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.823 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295531+0000", 2024-09-15T01:33:07.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295531+0000", 2024-09-15T01:33:07.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295532+0000", 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295842+0000", 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031032899999999998 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295879+0000", 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7038e-05 2024-09-15T01:33:07.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295885+0000", 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.331e-06 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295965+0000", 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9587999999999998e-05 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.295975+0000", 2024-09-15T01:33:07.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0247e-05 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.296096+0000", 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012046 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.296106+0000", 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0380000000000001e-05 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.296194+0000", 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8166999999999998e-05 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.296205+0000", 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0878e-05 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.302428+0000", 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0062232420000000004 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.828 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=54, reqid=client.4477.0:516, at_version=55'44, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.074223+0000", 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.670463012999999, 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089159619999999995, 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074223+0000", 2024-09-15T01:33:07.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074223+0000", 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074224+0000", 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0860000000000001e-06 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074256+0000", 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2258000000000002e-05 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075603+0000", 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001347139 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075620+0000", 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7198999999999999e-05 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075708+0000", 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8177999999999995e-05 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075730+0000", 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1469000000000001e-05 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076062+0000", 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033251199999999998 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076077+0000", 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4523e-05 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076128+0000", 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1371e-05 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076153+0000", 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4349e-05 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083139+0000", 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0069858780000000001 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=55, reqid=client.4477.0:517, at_version=55'45, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.669153141000001, 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016315308000000001, 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.834 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075533+0000", 2024-09-15T01:33:07.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9199999999999999e-07 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075557+0000", 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3264e-05 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076900+0000", 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013436749999999999 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076916+0000", 2024-09-15T01:33:07.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5302999999999999e-05 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076970+0000", 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4265999999999998e-05 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076980+0000", 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0521e-05 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077104+0000", 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012322599999999999 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077118+0000", 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4742e-05 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077164+0000", 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5972e-05 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077228+0000", 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3291000000000001e-05 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.091848+0000", 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014620055999999999 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=56, reqid=client.4477.0:518, at_version=55'46, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:07.839 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.667831506000001, 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015201957, 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076855+0000", 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.057e-06 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076876+0000", 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0874e-05 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079870+0000", 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0029943259999999998 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079882+0000", 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2026e-05 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079958+0000", 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5263000000000001e-05 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079969+0000", 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.145e-05 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.080116+0000", 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014736600000000001 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.080133+0000", 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6776e-05 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082691+0000", 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025575229999999999 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082740+0000", 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9373000000000003e-05 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.092056+0000", 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093159230000000003 2024-09-15T01:33:07.844 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=57, reqid=client.4477.0:518, at_version=0'0, trim_to=0'0, roll_forward_to=55'46))", 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.660907779, 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.77065714699999999, 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083779+0000", 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083792+0000", 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3478e-05 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853926+0000", 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.77013421000000004 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853932+0000", 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2820000000000004e-06 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853992+0000", 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9886999999999998e-05 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854001+0000", 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3700000000000001e-06 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854114+0000", 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000112651 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854152+0000", 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8518e-05 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854203+0000", 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1288000000000001e-05 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854223+0000", 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9884000000000001e-05 2024-09-15T01:33:07.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854435+0000", 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021157399999999999 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 56/54 ECSubWrite(tid=228, reqid=client.4477.0:520, at_version=56'33, trim_to=0'0, roll_forward_to=55'31))", 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.073896+0000", 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.6707899499999996, 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0079226000000000001, 2024-09-15T01:33:07.850 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.073896+0000", 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.073896+0000", 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.073896+0000", 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1199999999999999e-07 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.073929+0000", 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3324000000000001e-05 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074016+0000", 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7323000000000004e-05 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074024+0000", 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5349999999999999e-06 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074093+0000", 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9442000000000004e-05 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074104+0000", 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0569e-05 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074386+0000", 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028172599999999999 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074404+0000", 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8138000000000002e-05 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074454+0000", 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9929999999999998e-05 2024-09-15T01:33:07.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074606+0000", 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015201200000000001 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.081818+0000", 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0072123889999999996 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.855 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s3 56/31 ECSubWrite(tid=115, reqid=osd.14.0:115, at_version=56'45, trim_to=0'0, roll_forward_to=54'41))", 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.079178+0000", 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.6655078200000002, 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093555189999999996, 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079178+0000", 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079178+0000", 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079179+0000", 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1299999999999998e-07 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079210+0000", 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1012000000000001e-05 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079243+0000", 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3686000000000002e-05 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079250+0000", 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5490000000000003e-06 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079312+0000", 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2397000000000006e-05 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079324+0000", 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1362999999999999e-05 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.081543+0000", 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022197779999999999 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082438+0000", 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00089462899999999998 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082489+0000", 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1326999999999998e-05 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082657+0000", 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016720600000000001 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.088533+0000", 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0058766590000000002 2024-09-15T01:33:07.860 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.861 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.861 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.861 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:07.861 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:07.861 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:07.861 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:33:07.862 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:07.803+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:07.906 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:07.904+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:07.958 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:07.958 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:07.958 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:07.958 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:07.968 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:33:08.007 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:08.006+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:08.064 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:08.064 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:08.064 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:08.065 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:08.065 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:08.072 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:33:08.075 INFO:teuthology.orchestra.run.smithi042.stderr:marked in osd.9. 2024-09-15T01:33:08.076 INFO:tasks.rados.rados.0.smithi175.stdout:234: done (0 left) 2024-09-15T01:33:08.077 INFO:tasks.rados.rados.0.smithi175.stdout:235: rollback oid 33 current snap is 18 2024-09-15T01:33:08.077 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 33 to 15 2024-09-15T01:33:08.077 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 18 current snap is 18 2024-09-15T01:33:08.077 INFO:tasks.rados.rados.0.smithi175.stdout:236: seq_num 115 ranges {294912=770048,1064960=704512,1769472=458752} 2024-09-15T01:33:08.085 INFO:tasks.rados.rados.0.smithi175.stdout:236: writing smithi17529310-18 from 294912 to 1064960 tid 1 2024-09-15T01:33:08.090 INFO:tasks.rados.rados.0.smithi175.stdout:236: writing smithi17529310-18 from 1064960 to 1769472 tid 2 2024-09-15T01:33:08.094 INFO:tasks.rados.rados.0.smithi175.stdout:236: writing smithi17529310-18 from 1769472 to 2228224 tid 3 2024-09-15T01:33:08.094 INFO:tasks.rados.rados.0.smithi175.stdout:237: rollback oid 12 current snap is 18 2024-09-15T01:33:08.094 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 12 to 13 2024-09-15T01:33:08.094 INFO:tasks.rados.rados.0.smithi175.stdout:235: finishing rollback tid 0 to smithi17529310-33 2024-09-15T01:33:08.094 INFO:tasks.rados.rados.0.smithi175.stdout:235: finishing rollback tid 1 to smithi17529310-33 2024-09-15T01:33:08.094 INFO:tasks.rados.rados.0.smithi175.stdout:235: finishing rollback tid 2 to smithi17529310-33 2024-09-15T01:33:08.094 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 33 v 107 (ObjNum 104 snap 10 seq_num 104) dirty exists 2024-09-15T01:33:08.094 INFO:tasks.rados.rados.0.smithi175.stdout:235: done (2 left) 2024-09-15T01:33:08.094 INFO:tasks.rados.rados.0.smithi175.stdout:238: delete oid 30 current snap is 18 2024-09-15T01:33:08.097 INFO:tasks.rados.rados.0.smithi175.stdout:236: finishing write tid 1 to smithi17529310-18 2024-09-15T01:33:08.099 INFO:tasks.rados.rados.0.smithi175.stdout:237: finishing rollback tid 0 to smithi17529310-12 2024-09-15T01:33:08.100 INFO:tasks.rados.rados.0.smithi175.stdout:236: finishing write tid 2 to smithi17529310-18 2024-09-15T01:33:08.101 INFO:tasks.rados.rados.0.smithi175.stdout:236: finishing write tid 3 to smithi17529310-18 2024-09-15T01:33:08.101 INFO:tasks.rados.rados.0.smithi175.stdout:236: finishing write tid 4 to smithi17529310-18 2024-09-15T01:33:08.101 INFO:tasks.rados.rados.0.smithi175.stdout:237: finishing rollback tid 1 to smithi17529310-12 2024-09-15T01:33:08.101 INFO:tasks.rados.rados.0.smithi175.stdout:237: finishing rollback tid 2 to smithi17529310-12 2024-09-15T01:33:08.101 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 12 v 65 (ObjNum 98 snap 10 seq_num 98) dirty exists 2024-09-15T01:33:08.103 INFO:tasks.rados.rados.0.smithi175.stdout:236: finishing write tid 5 to smithi17529310-18 2024-09-15T01:33:08.103 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 18 v 59 (ObjNum 115 snap 18 seq_num 115) dirty exists 2024-09-15T01:33:08.103 INFO:tasks.rados.rados.0.smithi175.stdout:236: left oid 18 (ObjNum 115 snap 18 seq_num 115) 2024-09-15T01:33:08.104 INFO:tasks.thrashosds.thrasher:Added osd 9 2024-09-15T01:33:08.108 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:08.106+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:08.180 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:08.181 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:08.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:08.181 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:08.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.181 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:108 3.ds0 3:b063f44c:::smithi17529310-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:08.181 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:08.181 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.931541711000001, 2024-09-15T01:33:08.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013756292, 2024-09-15T01:33:08.181 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.181 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:08.181 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.181 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 108 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246459+0000", 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6800000000000001e-07 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246461+0000", 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.131e-06 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246468+0000", 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5529999999999998e-06 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246475+0000", 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5830000000000001e-06 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.248249+0000", 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017739990000000001 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.248333+0000", 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4288000000000003e-05 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.249154+0000", 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00082014900000000005 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.249218+0000", 2024-09-15T01:33:08.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4519999999999999e-05 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.260215+0000", 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010996601 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:187 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [rollback 3] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192368+0000", 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.985632640999999, 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010685266000000001, 2024-09-15T01:33:08.186 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 187 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192368+0000", 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192368+0000", 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192368+0000", 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3300000000000002e-07 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192372+0000", 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.36e-06 2024-09-15T01:33:08.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192384+0000", 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2481e-05 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192390+0000", 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4810000000000003e-06 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.193649+0000", 2024-09-15T01:33:08.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001258807 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.193722+0000", 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3140999999999996e-05 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194373+0000", 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00065084699999999997 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194463+0000", 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9424000000000003e-05 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194478+0000", 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5693999999999999e-05 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194534+0000", 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6008999999999999e-05 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194554+0000", 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9422000000000001e-05 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.196056+0000", 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001502506 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203053+0000", 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0069964609999999998 2024-09-15T01:33:08.192 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:188 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [append 0~0] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192497+0000", 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.985502831000002, 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010740421999999999, 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.193 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 188 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192497+0000", 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192497+0000", 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192498+0000", 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6899999999999997e-07 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192502+0000", 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.07e-06 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192512+0000", 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.039e-05 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192517+0000", 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7940000000000002e-06 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194593+0000", 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020753669999999998 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.194675+0000", 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2756000000000005e-05 2024-09-15T01:33:08.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195151+0000", 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047606299999999999 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195243+0000", 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1782999999999999e-05 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.195254+0000", 2024-09-15T01:33:08.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0908e-05 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.197131+0000", 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001876401 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.197156+0000", 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5364999999999999e-05 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:08.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.197402+0000", 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024628300000000003 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203238+0000", 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0058355730000000001 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:08.199 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.936777207999999, 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014051065, 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:08.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241243+0000", 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0126e-05 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241422+0000", 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017906899999999999 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241429+0000", 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4139999999999999e-06 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241500+0000", 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1098000000000003e-05 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241510+0000", 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8470000000000008e-06 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241787+0000", 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027729700000000003 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242007+0000", 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000220134 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242146+0000", 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013846799999999999 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242278+0000", 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0001326 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.255274+0000", 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01299575 2024-09-15T01:33:08.204 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.as1 44/38 ECSubWrite(tid=25, reqid=osd.3.0:25, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.241826+0000", 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.936174209000001, 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010494349, 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241826+0000", 2024-09-15T01:33:08.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241826+0000", 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241827+0000", 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2200000000000002e-07 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241852+0000", 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4672e-05 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241869+0000", 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7452000000000002e-05 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241876+0000", 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7390000000000002e-06 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242170+0000", 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029464500000000001 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242179+0000", 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4500000000000004e-06 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242421+0000", 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000242308 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242436+0000", 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4744e-05 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242608+0000", 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017215000000000001 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242622+0000", 2024-09-15T01:33:08.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4453999999999999e-05 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252320+0000", 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0096978129999999996 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s5 44/38 ECSubWrite(tid=20, reqid=osd.0.0:20, at_version=44'12, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.243644+0000", 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.934356469000001, 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012378171, 2024-09-15T01:33:08.210 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243644+0000", 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243644+0000", 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243644+0000", 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243670+0000", 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6384999999999999e-05 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243811+0000", 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014044900000000001 2024-09-15T01:33:08.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243817+0000", 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6680000000000004e-06 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244023+0000", 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020566099999999999 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244033+0000", 2024-09-15T01:33:08.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9510000000000001e-06 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245269+0000", 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001236109 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245301+0000", 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1911e-05 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245535+0000", 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023357599999999999 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245580+0000", 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5327999999999997e-05 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.256022+0000", 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010441884 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.215 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.934318863000001, 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014188612, 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:08.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243682+0000", 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0299999999999999e-07 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243699+0000", 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7575999999999999e-05 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244099+0000", 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039909500000000003 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244108+0000", 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3209999999999994e-06 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244168+0000", 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9688999999999997e-05 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244180+0000", 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2031000000000001e-05 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245504+0000", 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013241520000000001 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245537+0000", 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3287000000000002e-05 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245645+0000", 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108435 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245664+0000", 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8329999999999999e-05 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.257870+0000", 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012206092999999999 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 44/31 ECSubWrite(tid=152, reqid=osd.13.0:152, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.244552+0000", 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.933447901000001, 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010883719, 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244552+0000", 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244552+0000", 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244552+0000", 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8799999999999999e-07 2024-09-15T01:33:08.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244576+0000", 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3506000000000002e-05 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244641+0000", 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5073000000000005e-05 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244647+0000", 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4160000000000001e-06 2024-09-15T01:33:08.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244702+0000", 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4577999999999998e-05 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244713+0000", 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.084e-05 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244938+0000", 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022518700000000001 2024-09-15T01:33:08.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244968+0000", 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9748999999999999e-05 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245013+0000", 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5544000000000003e-05 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245032+0000", 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9091999999999999e-05 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.255436+0000", 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010403546 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:08.226 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.93103528, 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013797696999999999, 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246966+0000", 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000002e-07 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246979+0000", 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3111e-05 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247017+0000", 2024-09-15T01:33:08.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8096999999999998e-05 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247025+0000", 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9100000000000005e-06 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247095+0000", 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0049000000000006e-05 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247104+0000", 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7919999999999998e-06 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247217+0000", 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011299 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247376+0000", 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015968800000000001 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247417+0000", 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0228999999999997e-05 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248151+0000", 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00073475999999999999 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.260763+0000", 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012611282999999999 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.231 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.232 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.as1 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:08.232 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.247767+0000", 2024-09-15T01:33:08.232 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.930233434000002, 2024-09-15T01:33:08.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01354909, 2024-09-15T01:33:08.232 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.232 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.232 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247767+0000", 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247767+0000", 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247767+0000", 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4299999999999999e-07 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247778+0000", 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1027e-05 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247838+0000", 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9571000000000003e-05 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247842+0000", 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9740000000000004e-06 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247901+0000", 2024-09-15T01:33:08.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9694999999999997e-05 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247923+0000", 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1767e-05 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248086+0000", 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016259900000000001 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248105+0000", 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9609000000000001e-05 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248157+0000", 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1613999999999997e-05 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248189+0000", 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2140000000000001e-05 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.261316+0000", 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013126851 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s5 44/38 ECSubWrite(tid=21, reqid=osd.0.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.248138+0000", 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.929862603, 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013472511, 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.237 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248138+0000", 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248138+0000", 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248138+0000", 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4600000000000001e-07 2024-09-15T01:33:08.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248143+0000", 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6489999999999996e-06 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248368+0000", 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022486699999999999 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248375+0000", 2024-09-15T01:33:08.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0299999999999996e-06 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248429+0000", 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3236999999999999e-05 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248438+0000", 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8060000000000006e-06 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248547+0000", 2024-09-15T01:33:08.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108527 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248566+0000", 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9525999999999999e-05 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248606+0000", 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9730000000000001e-05 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248621+0000", 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4435e-05 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.261610+0000", 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012989458000000001 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 44/31 ECSubWrite(tid=153, reqid=osd.13.0:152, at_version=0'0, trim_to=0'0, roll_forward_to=44'17))", 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.248843+0000", 2024-09-15T01:33:08.242 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.92915674, 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013295741, 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248843+0000", 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248843+0000", 2024-09-15T01:33:08.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248844+0000", 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9599999999999999e-07 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248853+0000", 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.755e-06 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248917+0000", 2024-09-15T01:33:08.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3998999999999995e-05 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248923+0000", 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9139999999999996e-06 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248976+0000", 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3325000000000002e-05 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248986+0000", 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9799999999999993e-06 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.249081+0000", 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5148000000000004e-05 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.249121+0000", 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0111000000000003e-05 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.249145+0000", 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4247999999999999e-05 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.249158+0000", 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3063e-05 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.262139+0000", 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012980702 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.247 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:283 3.ds0 3:bfa066a3:::smithi17529310-39:head [copy-from ver 2] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.919234542000002, 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015911307999999999, 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 283 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.248 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258768+0000", 2024-09-15T01:33:08.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0980000000000001e-06 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258779+0000", 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1272e-05 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258795+0000", 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5679999999999999e-05 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258802+0000", 2024-09-15T01:33:08.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9029999999999998e-06 2024-09-15T01:33:08.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258879+0000", 2024-09-15T01:33:08.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7035999999999999e-05 2024-09-15T01:33:08.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.259139+0000", 2024-09-15T01:33:08.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260456 2024-09-15T01:33:08.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268247+0000", 2024-09-15T01:33:08.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0091076109999999998 2024-09-15T01:33:08.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268518+0000", 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027146499999999998 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268542+0000", 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3601e-05 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268616+0000", 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3944999999999998e-05 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.269087+0000", 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047095399999999999 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.274577+0000", 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0054903019999999999 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.274677+0000", 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9984999999999999e-05 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 44/31 ECSubWriteReply(tid=43, last_complete=44'32, committed=1, applied=1))", 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.902553676, 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.192594981, 2024-09-15T01:33:08.255 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.256 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.256 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:08.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:08.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275447+0000", 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2099999999999995e-07 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275452+0000", 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2050000000000004e-06 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.467889+0000", 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19243722999999999 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.467922+0000", 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3237e-05 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.468017+0000", 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4543999999999995e-05 2024-09-15T01:33:08.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.468041+0000", 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4844000000000001e-05 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:518 3.ds0 3:b50d79c0:::smithi17529310-18:head [append 0~0] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.073314+0000", 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.104686356, 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010662629999999999, 2024-09-15T01:33:08.259 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 518 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073314+0000", 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073314+0000", 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073314+0000", 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4000000000000002e-07 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073317+0000", 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3180000000000001e-06 2024-09-15T01:33:08.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073322+0000", 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4380000000000003e-06 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073325+0000", 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4829999999999999e-06 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075985+0000", 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026599309999999999 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076083+0000", 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7743000000000003e-05 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076539+0000", 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045620099999999999 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076644+0000", 2024-09-15T01:33:08.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104972 2024-09-15T01:33:08.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076752+0000", 2024-09-15T01:33:08.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107179 2024-09-15T01:33:08.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076979+0000", 2024-09-15T01:33:08.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000227347 2024-09-15T01:33:08.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077620+0000", 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00064142300000000001 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083336+0000", 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057156749999999999 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083976+0000", 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00064048000000000002 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.5s4 55/31 ECSubWrite(tid=49, reqid=client.4477.0:505, at_version=55'33, trim_to=0'0, roll_forward_to=54'31))", 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.082609+0000", 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.095391103000001, 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012942433999999999, 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082609+0000", 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082609+0000", 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082609+0000", 2024-09-15T01:33:08.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.17e-07 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082659+0000", 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0194000000000001e-05 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082708+0000", 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8046e-05 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082716+0000", 2024-09-15T01:33:08.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7269999999999996e-06 2024-09-15T01:33:08.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082772+0000", 2024-09-15T01:33:08.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5506e-05 2024-09-15T01:33:08.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082781+0000", 2024-09-15T01:33:08.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3549999999999994e-06 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083044+0000", 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00026238900000000002 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083106+0000", 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2136000000000003e-05 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083184+0000", 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8133000000000006e-05 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083201+0000", 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6884999999999998e-05 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.095552+0000", 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012350846 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.271 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 66/31 ECSubWrite(tid=241, reqid=osd.13.0:241, at_version=66'51, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081422+0000", 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.096578525999999998, 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013340902, 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081422+0000", 2024-09-15T01:33:08.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081422+0000", 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081422+0000", 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3300000000000001e-07 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081438+0000", 2024-09-15T01:33:08.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5982999999999999e-05 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081495+0000", 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7491999999999998e-05 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081502+0000", 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1450000000000002e-06 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081551+0000", 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8640000000000002e-05 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081560+0000", 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2410000000000001e-06 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081748+0000", 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018781399999999999 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083535+0000", 2024-09-15T01:33:08.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017866570000000001 2024-09-15T01:33:08.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083587+0000", 2024-09-15T01:33:08.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1956000000000002e-05 2024-09-15T01:33:08.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083618+0000", 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0858999999999997e-05 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.094763+0000", 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011144882 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 66/31 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=66'53, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:08.277 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.095318450999999998, 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014557652000000001, 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082683+0000", 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2999999999999999e-07 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082704+0000", 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1381000000000001e-05 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082736+0000", 2024-09-15T01:33:08.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2239000000000001e-05 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082743+0000", 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6449999999999999e-06 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082927+0000", 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018380800000000001 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082934+0000", 2024-09-15T01:33:08.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1600000000000001e-06 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083133+0000", 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019963000000000001 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083536+0000", 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040219000000000002 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083587+0000", 2024-09-15T01:33:08.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1600000000000001e-05 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083644+0000", 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6753999999999999e-05 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097239+0000", 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013595415 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.282 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.5s4 66/31 ECSubWrite(tid=54, reqid=osd.15.0:54, at_version=66'39, trim_to=0'0, roll_forward_to=56'37))", 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.084027+0000", 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.093973152000000004, 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012243765, 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084027+0000", 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084027+0000", 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084027+0000", 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.99e-07 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084051+0000", 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3657999999999999e-05 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084120+0000", 2024-09-15T01:33:08.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8998999999999994e-05 2024-09-15T01:33:08.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084128+0000", 2024-09-15T01:33:08.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9619999999999993e-06 2024-09-15T01:33:08.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084661+0000", 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000532866 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084670+0000", 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4939999999999992e-06 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084860+0000", 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000189543 2024-09-15T01:33:08.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085085+0000", 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022555399999999999 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085119+0000", 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4152000000000001e-05 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085139+0000", 2024-09-15T01:33:08.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9938000000000001e-05 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.096271+0000", 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0111314 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:609 3.ds0 3:b50d79c0:::smithi17529310-18:head [append 0~770048] snapc 1f=[1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.085186+0000", 2024-09-15T01:33:08.288 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.092813701999999998, 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013710833, 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 609 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085186+0000", 2024-09-15T01:33:08.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085186+0000", 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085188+0000", 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.694e-06 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085539+0000", 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035108399999999999 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085555+0000", 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6127999999999999e-05 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085565+0000", 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6819999999999998e-06 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085613+0000", 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7478999999999997e-05 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085720+0000", 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010775799999999999 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087279+0000", 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001558729 2024-09-15T01:33:08.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.092083+0000", 2024-09-15T01:33:08.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.004803501 2024-09-15T01:33:08.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.092092+0000", 2024-09-15T01:33:08.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3389999999999999e-06 2024-09-15T01:33:08.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.094255+0000", 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021635970000000002 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.096016+0000", 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017606130000000001 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.096085+0000", 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9275000000000003e-05 2024-09-15T01:33:08.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.098897+0000", 2024-09-15T01:33:08.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028119540000000002 2024-09-15T01:33:08.295 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.295 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.295 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.295 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.295 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.295 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:08.296 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:33:08.297 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:33:08.298 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:08.295+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:08.397 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:08.395+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:08.433 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:08.435 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:08.435 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:08.435 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:08.435 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:08.443 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:33:08.453 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:08.453 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:08.474 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:08.474 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:08.494 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:08.494 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:08.498 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:08.497+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:08.512 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:08.531 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:08.532 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:08.540 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:08.540 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:08.540 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:08.540 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:08.540 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:08.548 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:33:08.551 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:08.551 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:08.571 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:08.571 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:08.589 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:08.590 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:08.599 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:08.598+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:08.609 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:08.609 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:08.629 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:08.629 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:08.646 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:08.646 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:08.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:08.646 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:08.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.646 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:50 3.fs0 3:f8e751c8:::smithi17529310-17:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:08.646 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.189642+0000", 2024-09-15T01:33:08.646 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.453556999, 2024-09-15T01:33:08.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035043355999999998, 2024-09-15T01:33:08.646 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.646 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 50 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189642+0000", 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189642+0000", 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189643+0000", 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7399999999999998e-07 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189646+0000", 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2009999999999999e-06 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189652+0000", 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0340000000000002e-06 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189656+0000", 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.481e-06 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.190946+0000", 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0012900260000000001 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191064+0000", 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117945 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191469+0000", 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040500300000000003 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191570+0000", 2024-09-15T01:33:08.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101067 2024-09-15T01:33:08.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191588+0000", 2024-09-15T01:33:08.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7844999999999999e-05 2024-09-15T01:33:08.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191633+0000", 2024-09-15T01:33:08.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5028000000000003e-05 2024-09-15T01:33:08.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191816+0000", 2024-09-15T01:33:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018250699999999999 2024-09-15T01:33:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.224605+0000", 2024-09-15T01:33:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032789051999999999 2024-09-15T01:33:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.224685+0000", 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0692999999999995e-05 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:51 3.fs0 3:f8e751c8:::smithi17529310-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.189675+0000", 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.453523977000003, 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.036634376000000003, 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:08.653 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 51 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189675+0000", 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189675+0000", 2024-09-15T01:33:08.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189676+0000", 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0900000000000002e-07 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189678+0000", 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9889999999999999e-06 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189684+0000", 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5309999999999998e-06 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189688+0000", 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6610000000000002e-06 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191634+0000", 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019467029999999999 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.191709+0000", 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4474999999999994e-05 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.224734+0000", 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033024829999999998 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.224781+0000", 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6964e-05 2024-09-15T01:33:08.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.226309+0000", 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001528714 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:99 3.fs0 3:f630fe6e:::smithi17529310-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.407996578000002, 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055995037999999997, 2024-09-15T01:33:08.658 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235203+0000", 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4000000000000002e-07 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235205+0000", 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3e-06 2024-09-15T01:33:08.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235212+0000", 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4389999999999997e-06 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235216+0000", 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.031e-06 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.236877+0000", 2024-09-15T01:33:08.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00166088 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.236933+0000", 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6873999999999998e-05 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255906+0000", 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018972066999999999 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255957+0000", 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1467e-05 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291198+0000", 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035240540000000001 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:113 3.fs0 3:f8072b57:::smithi17529310-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.253601+0000", 2024-09-15T01:33:08.663 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.389597672000001, 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.037026959999999998, 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 113 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253601+0000", 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253601+0000", 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253602+0000", 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8800000000000003e-07 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253606+0000", 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1930000000000003e-06 2024-09-15T01:33:08.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253617+0000", 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1174e-05 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253621+0000", 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6009999999999999e-06 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.254591+0000", 2024-09-15T01:33:08.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000970114 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.254714+0000", 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012272300000000001 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255159+0000", 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044522399999999998 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255354+0000", 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000194666 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255360+0000", 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5359999999999998e-06 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255397+0000", 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6613000000000003e-05 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255520+0000", 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123161 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290561+0000", 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035041471999999997 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290628+0000", 2024-09-15T01:33:08.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6995000000000005e-05 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:114 3.fs0 3:f8072b57:::smithi17529310-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.389556657, 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048699039999999999, 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 114 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253643+0000", 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6899999999999998e-07 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253646+0000", 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253652+0000", 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3369999999999996e-06 2024-09-15T01:33:08.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253656+0000", 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.031e-06 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255410+0000", 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017543420000000001 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255505+0000", 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4704999999999995e-05 2024-09-15T01:33:08.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290833+0000", 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035327711999999997 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290893+0000", 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0384999999999999e-05 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.302341+0000", 2024-09-15T01:33:08.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011448274 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpReadReply(3.fs0 34/31 ECSubReadReply(tid=23, attrs_read=0))", 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.256270+0000", 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.386929023999997, 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034934552000000001, 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.256270+0000", 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.256270+0000", 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.256270+0000", 2024-09-15T01:33:08.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1800000000000001e-07 2024-09-15T01:33:08.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.256280+0000", 2024-09-15T01:33:08.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1900000000000001e-06 2024-09-15T01:33:08.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291049+0000", 2024-09-15T01:33:08.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034769125999999997 2024-09-15T01:33:08.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291054+0000", 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7990000000000001e-06 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291107+0000", 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2939999999999998e-05 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291205+0000", 2024-09-15T01:33:08.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8078999999999994e-05 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:145 3.cs0 3:34a45b5f:::smithi17529310-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.345330089000001, 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15351202899999999, 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 145 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0500000000000004e-07 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297871+0000", 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9769999999999999e-06 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297878+0000", 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9120000000000001e-06 2024-09-15T01:33:08.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297882+0000", 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0910000000000003e-06 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.298932+0000", 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00104904 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.299009+0000", 2024-09-15T01:33:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7095000000000006e-05 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.306936+0000", 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0079278730000000002 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.307022+0000", 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5254999999999996e-05 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.451381+0000", 2024-09-15T01:33:08.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14435928100000001 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=81, last_complete=44'30, committed=1, applied=1))", 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.242579+0000", 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.400619665000001, 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033297823999999997, 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242579+0000", 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242579+0000", 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242580+0000", 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3499999999999995e-07 2024-09-15T01:33:08.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242585+0000", 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8400000000000002e-06 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275688+0000", 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033103399999999998 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275692+0000", 2024-09-15T01:33:08.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9349999999999996e-06 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275708+0000", 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5930000000000002e-05 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275877+0000", 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016898400000000001 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.688 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:312 3.cs0 3:37dffec7:::smithi17529310-13:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:08.688 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:08.688 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.365006474000001, 2024-09-15T01:33:08.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770305000000002, 2024-09-15T01:33:08.688 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.688 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:08.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.688 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 312 2024-09-15T01:33:08.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.688 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0200000000000002e-07 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278196+0000", 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7319999999999998e-06 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278216+0000", 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0375999999999999e-05 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278221+0000", 2024-09-15T01:33:08.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4889999999999997e-06 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279668+0000", 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014473929999999999 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279753+0000", 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4567999999999994e-05 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280213+0000", 2024-09-15T01:33:08.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00046030399999999998 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280327+0000", 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011404399999999999 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280342+0000", 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4403000000000001e-05 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280467+0000", 2024-09-15T01:33:08.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000125594 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280562+0000", 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5317999999999998e-05 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328870+0000", 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048307782 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328963+0000", 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2700000000000004e-05 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:313 3.cs0 3:37dffec7:::smithi17529310-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.364973592999998, 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053267040000000002, 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 313 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278226+0000", 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7100000000000002e-07 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278228+0000", 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1859999999999999e-06 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278235+0000", 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9020000000000002e-06 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278240+0000", 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9540000000000003e-06 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280404+0000", 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021641469999999999 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280503+0000", 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9090000000000002e-05 2024-09-15T01:33:08.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328996+0000", 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048492533999999997 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.329053+0000", 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6997999999999999e-05 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.331493+0000", 2024-09-15T01:33:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024397580000000002 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=95, last_complete=44'34, committed=1, applied=1))", 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.361569249999999, 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047344538999999998, 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7700000000000001e-07 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281632+0000", 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.562e-06 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328775+0000", 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047143813 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328779+0000", 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.895e-06 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328803+0000", 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4076000000000001e-05 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328974+0000", 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017101599999999999 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.702 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:324 3.cs0 3:34a45b5f:::smithi17529310-48:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.312554143, 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18862163400000001, 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 324 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:08.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699999999999999e-07 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330648+0000", 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.694e-06 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330654+0000", 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4090000000000002e-06 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330659+0000", 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3210000000000004e-06 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332198+0000", 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015388699999999999 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332262+0000", 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4196000000000007e-05 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332785+0000", 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052292099999999998 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332870+0000", 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5049999999999994e-05 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332877+0000", 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7769999999999997e-06 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332902+0000", 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5012e-05 2024-09-15T01:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332962+0000", 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0649999999999997e-05 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519220+0000", 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18625739199999999 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519267+0000", 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6875000000000001e-05 2024-09-15T01:33:08.708 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:325 3.cs0 3:34a45b5f:::smithi17529310-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.312536080000001, 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.189890015, 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.709 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 325 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330667+0000", 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4410000000000002e-06 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330673+0000", 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5429999999999999e-06 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330676+0000", 2024-09-15T01:33:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.683e-06 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333439+0000", 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002763415 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333492+0000", 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2760000000000003e-05 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519496+0000", 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18600408199999999 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519560+0000", 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3660999999999999e-05 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.520553+0000", 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00099301399999999996 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=100, last_complete=44'39, committed=1, applied=1))", 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.310134037000001, 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18640773799999999, 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5999999999999999e-08 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333066+0000", 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.043e-06 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519088+0000", 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18602205399999999 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519092+0000", 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0890000000000002e-06 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519171+0000", 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8857999999999996e-05 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519473+0000", 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030159799999999998 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 47/31 ECSubWriteReply(tid=113, last_complete=47'51, committed=1, applied=1))", 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.383312226000001, 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.63648575600000001, 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.717 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:08.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-07 2024-09-15T01:33:08.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259890+0000", 2024-09-15T01:33:08.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5859999999999999e-06 2024-09-15T01:33:08.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.895924+0000", 2024-09-15T01:33:08.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.63603432699999995 2024-09-15T01:33:08.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.895929+0000", 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1340000000000003e-06 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.896000+0000", 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0687999999999998e-05 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.896373+0000", 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037265200000000001 2024-09-15T01:33:08.720 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 48/31 ECSubWriteReply(tid=128, last_complete=48'57, committed=1, applied=1))", 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.336496737000001, 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18111660500000001, 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306703+0000", 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3e-07 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306707+0000", 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7500000000000001e-06 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487741+0000", 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18103418199999999 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487745+0000", 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2189999999999998e-06 2024-09-15T01:33:08.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487789+0000", 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3430000000000003e-05 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487819+0000", 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0394000000000001e-05 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 53/31 ECSubWriteReply(tid=146, last_complete=53'74, committed=1, applied=1))", 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.442655287999999, 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.74248160299999999, 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-07 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200550+0000", 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5910000000000003e-06 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.942942+0000", 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.74239190499999996 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.942965+0000", 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2983999999999999e-05 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.943000+0000", 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5336999999999997e-05 2024-09-15T01:33:08.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.943025+0000", 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5250999999999999e-05 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.7s6 55/52 [PushOp(3:e6e8875f:::smithi17529310-31:head, version: 50'31, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:e6e8875f:::smithi17529310-31:head@50'31, size: 1949696, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.544940683, 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18071695199999999, 2024-09-15T01:33:08.728 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098259+0000", 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0200000000000001e-07 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098545+0000", 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028574699999999999 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278693+0000", 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.180148009 2024-09-15T01:33:08.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278700+0000", 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.888e-06 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278803+0000", 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102766 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278975+0000", 2024-09-15T01:33:08.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017284000000000001 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:524 3.fs0 3:f8e751c8:::smithi17529310-17:head [setxattr _header (57) in=7b] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.549176210000001, 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.115556992, 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.732 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 524 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2200000000000004e-07 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094028+0000", 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9710000000000003e-06 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094040+0000", 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1066e-05 2024-09-15T01:33:08.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094044+0000", 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8770000000000001e-06 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095122+0000", 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010771839999999999 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095188+0000", 2024-09-15T01:33:08.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6298999999999997e-05 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095521+0000", 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033339600000000002 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095614+0000", 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2295999999999999e-05 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:08.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095620+0000", 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8000000000000001e-06 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095653+0000", 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2867000000000002e-05 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095740+0000", 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6300999999999999e-05 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:08.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209497+0000", 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.113757184 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209580+0000", 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3128999999999996e-05 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:525 3.fs0 3:f8e751c8:::smithi17529310-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:08.738 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.549150385000001, 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11664384, 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 525 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6100000000000001e-07 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:08.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094051+0000", 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1950000000000002e-06 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094058+0000", 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4350000000000002e-06 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094063+0000", 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2039999999999996e-06 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095664+0000", 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001600642 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095725+0000", 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1707999999999999e-05 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209923+0000", 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.114197744 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209997+0000", 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3341000000000001e-05 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.210693+0000", 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00069601000000000005 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:08.743 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:08.744 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:08.745 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:08.745 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:08.746 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:08.746 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-15T01:33:08.746 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:08.743+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:08.747 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:08.747 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:08.747 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:08.748 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:08.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:08.705+0000 7fb22bfff640 0 -- 172.21.15.42:0/3293202490 >> [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] conn(0x7fb22c195d70 msgr2=0x7fb22c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:08.846 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:08.844+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:08.885 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:08.886 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:08.886 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:08.886 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:08.893 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-15T01:33:08.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:08.980 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:08.979+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:08.991 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:08.991 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:08.999 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-15T01:33:09.048 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:09.045+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:09.089 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T01:33:09.097 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:09.097 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:09.106 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:33:09.148 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:09.146+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:09.235 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:09.235 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:09.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:33:09.249 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:09.247+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:09.345 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:09.345 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:09.350 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:09.348+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:09.354 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:33:09.450 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:09.449+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:09.455 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:09.455 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:09.463 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:33:09.551 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:09.550+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:09.565 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:09.565 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:09.573 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:33:09.652 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:09.650+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:09.673 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:09.673 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:09.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:33:09.753 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:09.751+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:09.782 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:09.782 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:09.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:33:09.853 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:09.851+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:09.932 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:09.932 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:09.940 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:33:09.954 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:09.952+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:10.041 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:10.041 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:10.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:33:10.055 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:10.053+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:10.082 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-15T01:33:10.156 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:10.156 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:10.156 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:10.154+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:10.164 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-15T01:33:10.257 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:10.256+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:10.264 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:10.264 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:10.272 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-15T01:33:10.358 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:10.357+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:10.373 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:10.373 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:10.381 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-15T01:33:10.459 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:10.458+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:10.482 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:10.482 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:10.491 INFO:tasks.rados.rados.0.smithi175.stdout:236: done (2 left) 2024-09-15T01:33:10.491 INFO:tasks.rados.rados.0.smithi175.stdout:237: done (1 left) 2024-09-15T01:33:10.491 INFO:tasks.rados.rados.0.smithi175.stdout:238: done (0 left) 2024-09-15T01:33:10.491 INFO:tasks.rados.rados.0.smithi175.stdout:239: delete oid 38 current snap is 18 2024-09-15T01:33:10.494 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:33:10.494 INFO:tasks.rados.rados.0.smithi175.stdout:239: done (0 left) 2024-09-15T01:33:10.494 INFO:tasks.rados.rados.0.smithi175.stdout:240: snap_create 2024-09-15T01:33:10.560 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:10.557+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:10.603 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:10.603 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:10.611 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:33:10.661 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:10.660+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:10.749 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:10.749 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:10.757 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:33:10.762 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:10.761+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:10.861 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:33:10.862 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:10.861+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:10.940 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:33:10.957 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:10.957 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:10.964 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:10.962+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:10.965 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:33:11.064 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:11.064 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:11.064 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:11.063+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:11.071 INFO:tasks.rados.rados.0.smithi175.stdout:240: done (0 left) 2024-09-15T01:33:11.071 INFO:tasks.rados.rados.0.smithi175.stdout:241: read oid 48 snap -1 2024-09-15T01:33:11.071 INFO:tasks.rados.rados.0.smithi175.stdout:241: expect (ObjNum 84 snap 7 seq_num 84) 2024-09-15T01:33:11.072 INFO:tasks.rados.rados.0.smithi175.stdout:242: setattr oid 31 current snap is 19 2024-09-15T01:33:11.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:33:11.073 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:11.074 INFO:tasks.rados.rados.0.smithi175.stdout:243: read oid 42 snap -1 2024-09-15T01:33:11.074 INFO:tasks.rados.rados.0.smithi175.stdout:243: expect (ObjNum 105 snap 10 seq_num 105) 2024-09-15T01:33:11.074 INFO:tasks.rados.rados.0.smithi175.stdout:244: read oid 10 snap -1 2024-09-15T01:33:11.074 INFO:tasks.rados.rados.0.smithi175.stdout:244: expect (ObjNum 58 snap 3 seq_num 58) 2024-09-15T01:33:11.074 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 29 current snap is 19 2024-09-15T01:33:11.074 INFO:tasks.rados.rados.0.smithi175.stdout:245: seq_num 118 ranges {1622016=442368,2064384=753664,2818048=655360,3473408=147456} 2024-09-15T01:33:11.084 INFO:tasks.rados.rados.0.smithi175.stdout:245: writing smithi17529310-29 from 1622016 to 2064384 tid 1 2024-09-15T01:33:11.089 INFO:tasks.rados.rados.0.smithi175.stdout:245: writing smithi17529310-29 from 2064384 to 2818048 tid 2 2024-09-15T01:33:11.094 INFO:tasks.rados.rados.0.smithi175.stdout:245: writing smithi17529310-29 from 2818048 to 3473408 tid 3 2024-09-15T01:33:11.095 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:11.095 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:11.096 INFO:tasks.rados.rados.0.smithi175.stdout:245: writing smithi17529310-29 from 3473408 to 3620864 tid 4 2024-09-15T01:33:11.096 INFO:tasks.rados.rados.0.smithi175.stdout:246: copy_from oid 6 from oid 26 current snap is 19 2024-09-15T01:33:11.096 INFO:tasks.rados.rados.0.smithi175.stdout:247: delete oid 38 current snap is 19 2024-09-15T01:33:11.100 INFO:tasks.rados.rados.0.smithi175.stdout:245: finishing write tid 1 to smithi17529310-29 2024-09-15T01:33:11.100 INFO:tasks.rados.rados.0.smithi175.stdout:245: finishing write tid 2 to smithi17529310-29 2024-09-15T01:33:11.100 INFO:tasks.rados.rados.0.smithi175.stdout:245: finishing write tid 3 to smithi17529310-29 2024-09-15T01:33:11.101 INFO:tasks.rados.rados.0.smithi175.stdout:245: finishing write tid 4 to smithi17529310-29 2024-09-15T01:33:11.103 INFO:tasks.rados.rados.0.smithi175.stdout:245: finishing write tid 5 to smithi17529310-29 2024-09-15T01:33:11.106 INFO:tasks.rados.rados.0.smithi175.stdout:241: expect (ObjNum 84 snap 7 seq_num 84) 2024-09-15T01:33:11.115 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:11.116 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:11.135 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:11.135 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:11.153 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:11.153 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:11.165 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:11.162+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:11.172 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:11.179 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:33:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:33:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:33:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:36 3.8s0 3:1c3041d9:::smithi17529310-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.186121+0000", 2024-09-15T01:33:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.989587473, 2024-09-15T01:33:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0086069019999999996, 2024-09-15T01:33:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 36 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186121+0000", 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186121+0000", 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186122+0000", 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3200000000000004e-07 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186126+0000", 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.58e-06 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186135+0000", 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8969999999999997e-06 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.186139+0000", 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8439999999999997e-06 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188124+0000", 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019841070000000001 2024-09-15T01:33:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188203+0000", 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.193776+0000", 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005573153 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.193859+0000", 2024-09-15T01:33:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2838999999999997e-05 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.194728+0000", 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00086942300000000003 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:208 3.8s0 3:1226c857:::smithi17529310-32:head [setxattr _header (56) in=7b] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.219870+0000", 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.955838775, 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93580432499999999, 2024-09-15T01:33:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 208 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219870+0000", 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219870+0000", 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219871+0000", 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9100000000000002e-07 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219875+0000", 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7340000000000002e-06 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219882+0000", 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1729999999999998e-06 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219886+0000", 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5129999999999998e-06 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220853+0000", 2024-09-15T01:33:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096616900000000001 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220951+0000", 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8140000000000006e-05 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221294+0000", 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034291600000000002 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221386+0000", 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2517999999999997e-05 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221394+0000", 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8299999999999996e-06 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.223718+0000", 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023245449999999999 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.223730+0000", 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1659e-05 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.155614+0000", 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93188401099999996 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.155674+0000", 2024-09-15T01:33:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0226000000000002e-05 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:209 3.8s0 3:1226c857:::smithi17529310-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.219933+0000", 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.955775529, 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.937063969, 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 209 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219933+0000", 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219933+0000", 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219934+0000", 2024-09-15T01:33:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999999e-07 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219937+0000", 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3230000000000002e-06 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219949+0000", 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1205e-05 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219956+0000", 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1829999999999997e-06 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.222023+0000", 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020672009999999998 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.222072+0000", 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8752999999999999e-05 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.155960+0000", 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93388811299999996 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.156034+0000", 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4264999999999995e-05 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.156997+0000", 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096320599999999998 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:295 3.8s0 3:1226c857:::smithi17529310-32:head [rollback 8] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.269615+0000", 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.906094129, 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0098095839999999997, 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 295 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269615+0000", 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269615+0000", 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269615+0000", 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269618+0000", 2024-09-15T01:33:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1599999999999998e-06 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269630+0000", 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1532000000000001e-05 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269634+0000", 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1529999999999999e-06 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270497+0000", 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000863162 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270551+0000", 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4046999999999999e-05 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270862+0000", 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031048399999999998 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270964+0000", 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010242 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270976+0000", 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271008+0000", 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2227999999999997e-05 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271543+0000", 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00053470700000000004 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.277623+0000", 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060804329999999997 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.279424+0000", 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018007920000000001 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:296 3.8s0 3:1226c857:::smithi17529310-32:head [append 0~0] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.269639+0000", 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.906069759000001, 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011983048, 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 296 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269639+0000", 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269639+0000", 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269639+0000", 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269641+0000", 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0810000000000001e-06 2024-09-15T01:33:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269647+0000", 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8080000000000001e-06 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269652+0000", 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3470000000000001e-06 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271053+0000", 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014012899999999999 2024-09-15T01:33:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271118+0000", 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5505000000000004e-05 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271424+0000", 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030509800000000001 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271507+0000", 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3138000000000003e-05 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271523+0000", 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6511000000000002e-05 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271615+0000", 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1625000000000005e-05 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271647+0000", 2024-09-15T01:33:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2367999999999999e-05 2024-09-15T01:33:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.277790+0000", 2024-09-15T01:33:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006142659 2024-09-15T01:33:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.281622+0000", 2024-09-15T01:33:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038322790000000001 2024-09-15T01:33:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s5 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.272282+0000", 2024-09-15T01:33:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.903426674999999, 2024-09-15T01:33:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0098323179999999996, 2024-09-15T01:33:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272282+0000", 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272282+0000", 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272283+0000", 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4099999999999998e-07 2024-09-15T01:33:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272299+0000", 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6659000000000001e-05 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272328+0000", 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8515000000000001e-05 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272334+0000", 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272369+0000", 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5559999999999998e-05 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272379+0000", 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6809999999999993e-06 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272468+0000", 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8875999999999994e-05 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272473+0000", 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1200000000000001e-06 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272511+0000", 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8408999999999998e-05 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272521+0000", 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6309999999999998e-06 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.282114+0000", 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0095934420000000006 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s5 44/38 ECSubWrite(tid=62, reqid=client.4477.0:308, at_version=44'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.273293+0000", 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.902415781999999, 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089145920000000007, 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273293+0000", 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273293+0000", 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273293+0000", 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4900000000000001e-07 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273305+0000", 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1080999999999999e-05 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273325+0000", 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0196000000000001e-05 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273330+0000", 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273364+0000", 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4158000000000001e-05 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273370+0000", 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273422+0000", 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2333000000000002e-05 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273428+0000", 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1780000000000002e-06 2024-09-15T01:33:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273451+0000", 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3156e-05 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273464+0000", 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3654e-05 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.282208+0000", 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0087432849999999999 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=51, last_complete=47'39, committed=1, applied=1))", 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:49.360072+0000", 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.815637313, 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.83144309199999999, 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.360072+0000", 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.360072+0000", 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.360072+0000", 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7199999999999998e-07 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.360075+0000", 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7700000000000002e-06 2024-09-15T01:33:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.190654+0000", 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.83057901599999995 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.190662+0000", 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5650000000000004e-06 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.190823+0000", 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016112799999999999 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.191515+0000", 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00069194099999999995 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=52, last_complete=47'39, committed=1, applied=1))", 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:50.191812+0000", 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.983896621, 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047803646999999998, 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.191812+0000", 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.191812+0000", 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.191813+0000", 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8200000000000001e-07 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.191816+0000", 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0230000000000001e-06 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.239494+0000", 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047677905999999999 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.239499+0000", 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1619999999999999e-06 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.239592+0000", 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3135999999999997e-05 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.239616+0000", 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4037999999999999e-05 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:459 3.8s0 3:1c3041d9:::smithi17529310-12:head [setxattr _header (57) in=7b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:56.090711+0000", 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.084997447999999, 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.83095945999999998, 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 459 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090711+0000", 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090711+0000", 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090712+0000", 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3799999999999997e-07 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090715+0000", 2024-09-15T01:33:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.19e-06 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090731+0000", 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6305999999999999e-05 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090739+0000", 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5580000000000003e-06 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.093526+0000", 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027874839999999998 2024-09-15T01:33:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.093626+0000", 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9702000000000002e-05 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094042+0000", 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041598299999999998 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094169+0000", 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012725799999999999 2024-09-15T01:33:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094178+0000", 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0100000000000001e-06 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094213+0000", 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4171000000000002e-05 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094316+0000", 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103417 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.921612+0000", 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.82729596100000002 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.921671+0000", 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8882000000000003e-05 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:460 3.8s0 3:1c3041d9:::smithi17529310-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:56.090744+0000", 2024-09-15T01:33:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.084965109000001, 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.83214597000000001, 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 460 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090744+0000", 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090744+0000", 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090744+0000", 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3499999999999996e-07 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090747+0000", 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8930000000000001e-06 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090755+0000", 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0420000000000003e-06 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090760+0000", 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7779999999999998e-06 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094227+0000", 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003466883 2024-09-15T01:33:11.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094296+0000", 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9388999999999996e-05 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.921916+0000", 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.82761973799999999 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:11.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.922012+0000", 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5470000000000006e-05 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.922890+0000", 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00087814200000000005 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:503 3.8s0 3:1abde0c6:::smithi17529310-42:head [copy-from ver 41] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.073001+0000", 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.102708184000001, 2024-09-15T01:33:11.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032844862000000002, 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 503 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073001+0000", 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073001+0000", 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073003+0000", 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9690000000000001e-06 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073009+0000", 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1299999999999998e-06 2024-09-15T01:33:11.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073023+0000", 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4684999999999999e-05 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073033+0000", 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2750000000000001e-06 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073097+0000", 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4503000000000002e-05 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073188+0000", 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0748000000000006e-05 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.095640+0000", 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022451861 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096520+0000", 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00088040899999999997 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096558+0000", 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7358999999999999e-05 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096970+0000", 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041243600000000002 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096989+0000", 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9110000000000002e-05 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.105798+0000", 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0088090970000000001 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.105846+0000", 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7280000000000001e-05 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:504 3.8s0 3:1abde0c6:::smithi17529310-42:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.073045+0000", 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.102663701999999, 2024-09-15T01:33:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023923209000000001, 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 504 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073045+0000", 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073045+0000", 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073046+0000", 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0100000000000005e-07 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073049+0000", 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2619999999999999e-06 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073065+0000", 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5794000000000002e-05 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073070+0000", 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4729999999999999e-06 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073328+0000", 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025764599999999999 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073406+0000", 2024-09-15T01:33:11.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7825999999999997e-05 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096769+0000", 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023363063 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096883+0000", 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114212 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096968+0000", 2024-09-15T01:33:11.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5332000000000002e-05 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs6 55/31 ECSubWrite(tid=160, reqid=client.4477.0:512, at_version=55'53, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.075658+0000", 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.100050438, 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0092545209999999999, 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075658+0000", 2024-09-15T01:33:11.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075658+0000", 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075659+0000", 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6199999999999997e-07 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075674+0000", 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4739000000000001e-05 2024-09-15T01:33:11.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075705+0000", 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0618999999999999e-05 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075711+0000", 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2179999999999996e-06 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075753+0000", 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2178999999999997e-05 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075763+0000", 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7839999999999998e-06 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075863+0000", 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100163 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075874+0000", 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-05 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075913+0000", 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9539000000000002e-05 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075929+0000", 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5554999999999999e-05 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.084913+0000", 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089839229999999996 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.251 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs6 55/31 ECSubWrite(tid=161, reqid=client.4477.0:513, at_version=55'54, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.076290+0000", 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.099419130999999, 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0087137010000000008, 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.076290+0000", 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.076290+0000", 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.076291+0000", 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1960000000000001e-06 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.076311+0000", 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9913999999999999e-05 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.076351+0000", 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9792000000000001e-05 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.076358+0000", 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5700000000000004e-06 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.076405+0000", 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6777999999999998e-05 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.076414+0000", 2024-09-15T01:33:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8899999999999996e-06 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.076508+0000", 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4448000000000001e-05 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.076516+0000", 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5209999999999997e-06 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.076554+0000", 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8120000000000001e-05 2024-09-15T01:33:11.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.076570+0000", 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6427999999999999e-05 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.085003+0000", 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084330440000000007 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.256 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:550 3.8s0 3:1abde0c6:::smithi17529310-42:head [rollback 19] snapc 19=[19,15] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:01.086634+0000", 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.089074375999999, 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0096404579999999993, 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 550 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086634+0000", 2024-09-15T01:33:11.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086634+0000", 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086635+0000", 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7000000000000005e-07 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086638+0000", 2024-09-15T01:33:11.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3380000000000001e-06 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086646+0000", 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.763e-06 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086651+0000", 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.087875+0000", 2024-09-15T01:33:11.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001223745 2024-09-15T01:33:11.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:11.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.087950+0000", 2024-09-15T01:33:11.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5459999999999999e-05 2024-09-15T01:33:11.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.088528+0000", 2024-09-15T01:33:11.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0005775 2024-09-15T01:33:11.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.088644+0000", 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116746 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.088660+0000", 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5642999999999999e-05 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.088711+0000", 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1127e-05 2024-09-15T01:33:11.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.089434+0000", 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00072320700000000002 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.090185+0000", 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075082399999999998 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.096275+0000", 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060898439999999996 2024-09-15T01:33:11.262 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:551 3.8s0 3:1abde0c6:::smithi17529310-42:head [append 0~0] snapc 19=[19,15] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:01.086656+0000", 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.089053010000001, 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0097169549999999993, 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:11.263 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 551 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086656+0000", 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086656+0000", 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086656+0000", 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086659+0000", 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.339e-06 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086665+0000", 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0630000000000002e-06 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086669+0000", 2024-09-15T01:33:11.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9469999999999996e-06 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.088717+0000", 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020487539999999999 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.088808+0000", 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0724999999999996e-05 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.089265+0000", 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045738199999999999 2024-09-15T01:33:11.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.089355+0000", 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9184999999999994e-05 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.089367+0000", 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2371999999999999e-05 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.089636+0000", 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000269225 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.089844+0000", 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020802099999999999 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.090364+0000", 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051979399999999996 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.096373+0000", 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060087969999999997 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.268 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs6 59/31 ECSubWrite(tid=179, reqid=client.4477.0:552, at_version=59'58, trim_to=0'0, roll_forward_to=56'56))", 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:01.091327+0000", 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.084381818000001, 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0086150180000000003, 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.091327+0000", 2024-09-15T01:33:11.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.091327+0000", 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.091327+0000", 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.97e-07 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.091437+0000", 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010992099999999999 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.091472+0000", 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4477999999999998e-05 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.091488+0000", 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6283999999999999e-05 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.091550+0000", 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2283e-05 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.091561+0000", 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0893999999999999e-05 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.092173+0000", 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00061210099999999999 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.092182+0000", 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3979999999999999e-06 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.092714+0000", 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00053269700000000005 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.092734+0000", 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9544000000000001e-05 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.099942+0000", 2024-09-15T01:33:11.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072082209999999999 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.3s5 66/31 ECSubWrite(tid=241, reqid=osd.13.0:241, at_version=66'51, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:08.081466+0000", 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.094242967, 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0094524540000000008, 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081466+0000", 2024-09-15T01:33:11.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081466+0000", 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081466+0000", 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6199999999999999e-07 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081495+0000", 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8538999999999999e-05 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081542+0000", 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7707000000000002e-05 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081551+0000", 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2490000000000001e-06 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081607+0000", 2024-09-15T01:33:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5988e-05 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081617+0000", 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.023e-05 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083363+0000", 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017467559999999999 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083376+0000", 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2729e-05 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083398+0000", 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2192999999999999e-05 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083614+0000", 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021578999999999999 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.090918+0000", 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073041110000000003 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.3s5 66/31 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=66'53, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:08.082267+0000", 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.0934420880000002, 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0094909190000000004, 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.082267+0000", 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.082267+0000", 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.082267+0000", 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.082287+0000", 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9110000000000002e-05 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.082319+0000", 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2620000000000003e-05 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.082324+0000", 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083400+0000", 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001076013 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083407+0000", 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2199999999999997e-06 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083596+0000", 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018967699999999999 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083604+0000", 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4139999999999997e-06 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083644+0000", 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9824999999999999e-05 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083656+0000", 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2488e-05 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.091758+0000", 2024-09-15T01:33:11.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0081016390000000008 2024-09-15T01:33:11.284 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.284 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.284 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.284 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.284 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.284 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:11.285 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:11.285 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:11.286 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:11.286 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:11.286 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:33:11.287 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:11.283+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:11.287 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:11.287 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:11.288 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:11.288 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:11.288 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:11.288 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:11.288 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:11.289 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:11.304 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:11.304 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:11.323 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:11.323 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:11.343 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:11.344 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:11.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:11.342+0000 7f6c9207a640 0 -- 172.21.15.42:0/377504775 >> [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] conn(0x7f6c8c195790 msgr2=0x7f6c8c197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:11.386 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:11.384+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:11.393 INFO:tasks.rados.rados.0.smithi175.stdout:243: expect (ObjNum 105 snap 10 seq_num 105) 2024-09-15T01:33:11.424 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:33:11.424 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:11.432 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:33:11.487 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:11.485+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:11.514 INFO:tasks.rados.rados.0.smithi175.stdout:241: done (6 left) 2024-09-15T01:33:11.514 INFO:tasks.rados.rados.0.smithi175.stdout:243: done (5 left) 2024-09-15T01:33:11.514 INFO:tasks.rados.rados.0.smithi175.stdout:247: done (4 left) 2024-09-15T01:33:11.514 INFO:tasks.rados.rados.0.smithi175.stdout:248: delete oid 13 current snap is 19 2024-09-15T01:33:11.516 INFO:tasks.rados.rados.0.smithi175.stdout:248: done (4 left) 2024-09-15T01:33:11.516 INFO:tasks.rados.rados.0.smithi175.stdout:249: read oid 23 snap -1 2024-09-15T01:33:11.516 INFO:tasks.rados.rados.0.smithi175.stdout:249: expect (ObjNum 114 snap 17 seq_num 114) 2024-09-15T01:33:11.516 INFO:tasks.rados.rados.0.smithi175.stdout:250: setattr oid 12 current snap is 19 2024-09-15T01:33:11.517 INFO:tasks.rados.rados.0.smithi175.stdout:251: snap_create 2024-09-15T01:33:11.564 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:11.565 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:11.571 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:11.571 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:11.571 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:33:11.571 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:33:11.571 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:11.580 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:33:11.588 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:11.586+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:11.677 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:11.677 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:33:11.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:33:11.677 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:33:11.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 33/31 ECSubWrite(tid=2, reqid=client.4477.0:1, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.176343+0000", 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.498210464000003, 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096529600000000004, 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176343+0000", 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176343+0000", 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176343+0000", 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3999999999999998e-07 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176405+0000", 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1896000000000005e-05 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176489+0000", 2024-09-15T01:33:11.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3608000000000003e-05 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176500+0000", 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0685e-05 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176570+0000", 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0057999999999999e-05 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176580+0000", 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0309000000000001e-05 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176704+0000", 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012443600000000001 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176750+0000", 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5213999999999998e-05 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176809+0000", 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9477e-05 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176988+0000", 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178636 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177309+0000", 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032073699999999998 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 33/31 ECSubWrite(tid=3, reqid=client.4477.0:2, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.176414+0000", 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.498139919000003, 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010880270000000001, 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176414+0000", 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176414+0000", 2024-09-15T01:33:11.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176414+0000", 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7100000000000002e-07 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176430+0000", 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5682e-05 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176522+0000", 2024-09-15T01:33:11.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2359000000000001e-05 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176527+0000", 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176766+0000", 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000238411 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176802+0000", 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5769000000000003e-05 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176941+0000", 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013990500000000001 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.176969+0000", 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7251999999999999e-05 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177029+0000", 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9933999999999999e-05 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177048+0000", 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9123e-05 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.177502+0000", 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045398500000000001 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 34/31 ECSubWrite(tid=0, reqid=client.4477.0:13, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179137+0000", 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.495416355000003, 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017856180000000001, 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179137+0000", 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179137+0000", 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179138+0000", 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3900000000000004e-07 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179179+0000", 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0599999999999998e-05 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179867+0000", 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068798899999999996 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179876+0000", 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180320+0000", 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044354399999999998 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180328+0000", 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7649999999999993e-06 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180424+0000", 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6311000000000007e-05 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180437+0000", 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3001e-05 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180528+0000", 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1378000000000005e-05 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180623+0000", 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4455000000000003e-05 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180923+0000", 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030002300000000001 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.692 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=1, reqid=client.4477.0:28, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.495371143, 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018499090000000001, 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179245+0000", 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2150000000000006e-05 2024-09-15T01:33:11.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180346+0000", 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001101196 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180357+0000", 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175000000000001e-05 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180408+0000", 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0590999999999997e-05 2024-09-15T01:33:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180417+0000", 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8060000000000001e-06 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180553+0000", 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136324 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180684+0000", 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000130853 2024-09-15T01:33:11.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180745+0000", 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0971000000000003e-05 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180919+0000", 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017386800000000001 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181032+0000", 2024-09-15T01:33:11.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113578 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 34/31 ECSubWrite(tid=2, reqid=client.4477.0:14, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.495304808, 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018048179999999999, 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179259+0000", 2024-09-15T01:33:11.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0104e-05 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180590+0000", 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013307779999999999 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180598+0000", 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180692+0000", 2024-09-15T01:33:11.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3549999999999997e-05 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180700+0000", 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6330000000000002e-06 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180793+0000", 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967000000000006e-05 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180801+0000", 2024-09-15T01:33:11.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4580000000000004e-06 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180836+0000", 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5732999999999998e-05 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180858+0000", 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.126e-05 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181054+0000", 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019604099999999999 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 33/31 ECSubWrite(tid=4, reqid=client.4477.0:2, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179837+0000", 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.494716482999998, 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00111943, 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179837+0000", 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179837+0000", 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179838+0000", 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.096e-06 2024-09-15T01:33:11.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179853+0000", 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4919999999999999e-05 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179893+0000", 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9502999999999998e-05 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179902+0000", 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:33:11.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179998+0000", 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6417999999999997e-05 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180008+0000", 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8980000000000008e-06 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180107+0000", 2024-09-15T01:33:11.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9267999999999999e-05 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180121+0000", 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3715e-05 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180201+0000", 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0383999999999995e-05 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180934+0000", 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00073209700000000002 2024-09-15T01:33:11.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180957+0000", 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3104000000000001e-05 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 34/31 ECSubWrite(tid=4, reqid=client.4477.0:31, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.180672+0000", 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.493881399000003, 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014757450000000001, 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.708 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180672+0000", 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180672+0000", 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180673+0000", 2024-09-15T01:33:11.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2799999999999996e-07 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181385+0000", 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071188100000000004 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181419+0000", 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4078999999999997e-05 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181425+0000", 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5989999999999998e-06 2024-09-15T01:33:11.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181488+0000", 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2407000000000001e-05 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181498+0000", 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003999999999999e-05 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181638+0000", 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014010499999999999 2024-09-15T01:33:11.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181679+0000", 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0672e-05 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181759+0000", 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0067000000000004e-05 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181779+0000", 2024-09-15T01:33:11.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0336e-05 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182148+0000", 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000369067 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=0, reqid=client.4477.0:16, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.181909+0000", 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.492644992999999, 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00076923200000000001, 2024-09-15T01:33:11.713 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181909+0000", 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181909+0000", 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181909+0000", 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2500000000000001e-07 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181978+0000", 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8928000000000001e-05 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182070+0000", 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2204e-05 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182077+0000", 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0210000000000002e-06 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182122+0000", 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4582000000000001e-05 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182131+0000", 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8510000000000005e-06 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182261+0000", 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000129875 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182294+0000", 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3291999999999999e-05 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182356+0000", 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1696e-05 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182487+0000", 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013164000000000001 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.182678+0000", 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019071800000000001 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=1, reqid=client.4477.0:17, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.183115+0000", 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.491438602999999, 2024-09-15T01:33:11.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00064692799999999996, 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183115+0000", 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183115+0000", 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183116+0000", 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000003e-07 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183134+0000", 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8031000000000001e-05 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183179+0000", 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5466999999999997e-05 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183187+0000", 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1899999999999995e-06 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183249+0000", 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1975000000000002e-05 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183276+0000", 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7056e-05 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183365+0000", 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8863000000000006e-05 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183397+0000", 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2082999999999998e-05 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183439+0000", 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1269000000000001e-05 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183458+0000", 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9570000000000001e-05 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.183762+0000", 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030372599999999999 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 34/31 ECSubWrite(tid=6, reqid=client.4477.0:43, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.187004+0000", 2024-09-15T01:33:11.723 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.487549823000002, 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011234019999999999, 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187004+0000", 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187004+0000", 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187005+0000", 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0470000000000001e-06 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187114+0000", 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010932700000000001 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187177+0000", 2024-09-15T01:33:11.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2597999999999999e-05 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187185+0000", 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1780000000000003e-06 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187279+0000", 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3690999999999995e-05 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187288+0000", 2024-09-15T01:33:11.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2799999999999992e-06 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187629+0000", 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034080200000000002 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187682+0000", 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2848e-05 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187752+0000", 2024-09-15T01:33:11.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9906000000000005e-05 2024-09-15T01:33:11.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188095+0000", 2024-09-15T01:33:11.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000343297 2024-09-15T01:33:11.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188127+0000", 2024-09-15T01:33:11.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2428000000000002e-05 2024-09-15T01:33:11.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.728 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=2, reqid=client.4477.0:17, at_version=0'0, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.187733+0000", 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.486820299999998, 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010603889999999999, 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187733+0000", 2024-09-15T01:33:11.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187733+0000", 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188139+0000", 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040583200000000001 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188168+0000", 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8878999999999999e-05 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188207+0000", 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9385999999999999e-05 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188219+0000", 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1521e-05 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188274+0000", 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5324000000000001e-05 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188284+0000", 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8239999999999995e-06 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188391+0000", 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106909 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188449+0000", 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8193999999999999e-05 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188505+0000", 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6033999999999999e-05 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188615+0000", 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109959 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.188794+0000", 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017852699999999999 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=16, reqid=client.4477.0:76, at_version=34'5, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.214947+0000", 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.459606758, 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071814299999999997, 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.214947+0000", 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.214947+0000", 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.214948+0000", 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4810000000000001e-06 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215065+0000", 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116551 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215104+0000", 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9462000000000003e-05 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215113+0000", 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2689999999999999e-06 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215157+0000", 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4127e-05 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215163+0000", 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4670000000000001e-06 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215295+0000", 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013140999999999999 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215304+0000", 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0399999999999998e-06 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215379+0000", 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5007999999999997e-05 2024-09-15T01:33:11.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215395+0000", 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5897999999999999e-05 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.215665+0000", 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027043 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.739 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=4, reqid=client.4477.0:88, at_version=34'3, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.226933+0000", 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.447620874000002, 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010392190000000001, 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.226933+0000", 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.226933+0000", 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.226934+0000", 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.077e-06 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227240+0000", 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030573500000000001 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227270+0000", 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.021e-05 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227277+0000", 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1629999999999999e-06 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227326+0000", 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9227000000000001e-05 2024-09-15T01:33:11.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227335+0000", 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9609999999999994e-06 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227508+0000", 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017294000000000001 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227520+0000", 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1963e-05 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227623+0000", 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102663 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227724+0000", 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010162600000000001 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227972+0000", 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024765399999999998 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 34/31 ECSubWrite(tid=5, reqid=client.4477.0:89, at_version=34'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.227505+0000", 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.447049012000001, 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00064992299999999995, 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.745 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227505+0000", 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227505+0000", 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227505+0000", 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1800000000000005e-07 2024-09-15T01:33:11.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227523+0000", 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7708999999999999e-05 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227558+0000", 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4310000000000002e-05 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227564+0000", 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4690000000000002e-06 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227610+0000", 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6440000000000003e-05 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227620+0000", 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4029999999999996e-06 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227705+0000", 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5518000000000004e-05 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227713+0000", 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.627e-06 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227739+0000", 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6342999999999998e-05 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.227755+0000", 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6164e-05 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.228155+0000", 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039912200000000002 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=18, reqid=client.4477.0:91, at_version=34'7, trim_to=0'0, roll_forward_to=34'5))", 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.230472+0000", 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.444082082999998, 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00078053499999999995, 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230472+0000", 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230472+0000", 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230472+0000", 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7800000000000001e-07 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230677+0000", 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000204157 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230711+0000", 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4150999999999999e-05 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230718+0000", 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4869999999999996e-06 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230760+0000", 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1587e-05 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230768+0000", 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1640000000000001e-06 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230905+0000", 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000137269 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.230916+0000", 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1104000000000001e-05 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.231004+0000", 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7379999999999993e-05 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.231016+0000", 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1934e-05 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.231252+0000", 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236524 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 34/31 ECSubWrite(tid=6, reqid=client.4477.0:118, at_version=34'3, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.263014+0000", 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.411539435000002, 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001181788, 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263014+0000", 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263014+0000", 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263016+0000", 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.424e-06 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263549+0000", 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00053332100000000001 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263582+0000", 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2926000000000003e-05 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263591+0000", 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263633+0000", 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2271999999999998e-05 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263640+0000", 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8190000000000003e-06 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263822+0000", 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018233 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263831+0000", 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263930+0000", 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8944999999999995e-05 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263964+0000", 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4173e-05 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.264196+0000", 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023164099999999999 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 34/31 ECSubWrite(tid=7, reqid=client.4477.0:119, at_version=34'4, trim_to=0'0, roll_forward_to=33'2))", 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.263660+0000", 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.410893846, 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065152199999999997, 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.760 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263660+0000", 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263660+0000", 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263660+0000", 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0400000000000002e-07 2024-09-15T01:33:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263674+0000", 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3579e-05 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263697+0000", 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3499999999999999e-05 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263703+0000", 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-09-15T01:33:11.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263869+0000", 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016557099999999999 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263878+0000", 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263950+0000", 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496999999999994e-05 2024-09-15T01:33:11.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263956+0000", 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.305e-06 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263980+0000", 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4036000000000002e-05 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.263991+0000", 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1770000000000001e-05 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.264311+0000", 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031986800000000001 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 34/31 ECSubWrite(tid=23, reqid=client.4477.0:150, at_version=34'3, trim_to=0'0, roll_forward_to=34'1))", 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.307830+0000", 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.366723767000003, 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070952099999999996, 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.765 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307830+0000", 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307830+0000", 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307832+0000", 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2900000000000001e-06 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307956+0000", 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123354 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307991+0000", 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5608000000000002e-05 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.307999+0000", 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0790000000000001e-06 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308047+0000", 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7756000000000002e-05 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308054+0000", 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5050000000000004e-06 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308144+0000", 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0972999999999998e-05 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:11.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308152+0000", 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308239+0000", 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6976000000000002e-05 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308258+0000", 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8961e-05 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.308539+0000", 2024-09-15T01:33:11.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028192700000000001 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.bs6 37/36 ECSubRead(tid=38, to_read={3:df51d772:::smithi17529310-50:head=0,262144,0}, subchunks={3:df51d772:::smithi17529310-50:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:39.198209+0000", 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.476344744999999, 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092439400000000004, 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198209+0000", 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198209+0000", 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198209+0000", 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2e-07 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198215+0000", 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3499999999999996e-06 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198235+0000", 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0565e-05 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198240+0000", 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0540000000000002e-06 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.198279+0000", 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8664000000000001e-05 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.199133+0000", 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000854341 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.6s6 37/36 ECSubRead(tid=10, to_read={3:65d6c4b7:::smithi17529310-30:head=0,262144,0}, subchunks={3:65d6c4b7:::smithi17529310-30:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:39.201538+0000", 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.473015189999998, 2024-09-15T01:33:11.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011658199999999999, 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201538+0000", 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201538+0000", 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201539+0000", 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201546+0000", 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5699999999999998e-06 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201571+0000", 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5381e-05 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:11.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201576+0000", 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7790000000000003e-06 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.201618+0000", 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1999999999999998e-05 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:39.202704+0000", 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010864640000000001 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:11.777 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:11.777 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:11.778 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-15T01:33:11.778 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:11.776+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:11.875 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:33:11.875 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:11.878 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:11.877+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:11.883 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-15T01:33:11.884 INFO:tasks.rados.rados.0.smithi175.stdout:245: finishing write tid 6 to smithi17529310-29 2024-09-15T01:33:11.884 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 29 v 121 (ObjNum 118 snap 19 seq_num 118) dirty exists 2024-09-15T01:33:11.884 INFO:tasks.rados.rados.0.smithi175.stdout:245: left oid 29 (ObjNum 118 snap 19 seq_num 118) 2024-09-15T01:33:11.979 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:11.979 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:11.979 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:33:11.979 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:33:11.979 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:11.979 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:11.977+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:11.987 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-15T01:33:12.075 INFO:tasks.rados.rados.0.smithi175.stdout:245: done (6 left) 2024-09-15T01:33:12.075 INFO:tasks.rados.rados.0.smithi175.stdout:251: done (5 left) 2024-09-15T01:33:12.075 INFO:tasks.rados.rados.0.smithi175.stdout:252: snap_remove snap 18 2024-09-15T01:33:12.080 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:12.078+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:12.101 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:33:12.133 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:12.133 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:33:12.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:33:12.133 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:33:12.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.133 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:33:12.133 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:33:12.133 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.651141285999998, 2024-09-15T01:33:12.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27210968699999999, 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479166+0000", 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4729999999999999e-06 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479172+0000", 2024-09-15T01:33:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7460000000000003e-06 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479186+0000", 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4011e-05 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479211+0000", 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4658e-05 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479249+0000", 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7706000000000001e-05 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479255+0000", 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9669999999999998e-06 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750042+0000", 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27078760899999998 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:12.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750204+0000", 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016171800000000001 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750337+0000", 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000132674 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750470+0000", 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013314699999999999 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750485+0000", 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5288e-05 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750515+0000", 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0179e-05 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750582+0000", 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6890999999999994e-05 2024-09-15T01:33:12.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751212+0000", 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062971900000000005 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751249+0000", 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7072e-05 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751274+0000", 2024-09-15T01:33:12.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4828999999999999e-05 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:9 3.3s0 3:c07cd22b:::smithi17529310-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.175883+0000", 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.954422180999998, 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01603094, 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:12.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175883+0000", 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175883+0000", 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175884+0000", 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175886+0000", 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.429e-06 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175893+0000", 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6630000000000004e-06 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175900+0000", 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999997e-06 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178335+0000", 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024349200000000001 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178422+0000", 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7049e-05 2024-09-15T01:33:12.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187881+0000", 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0094586740000000002 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187983+0000", 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102005 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.191914+0000", 2024-09-15T01:33:12.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039315039999999997 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 34/31 ECSubWrite(tid=13, reqid=client.4477.0:128, at_version=34'8, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.852991494999998, 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.101528777, 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7999999999999995e-07 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277326+0000", 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.184e-05 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378494+0000", 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.101167374 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378498+0000", 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1150000000000004e-06 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378565+0000", 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7354999999999995e-05 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378574+0000", 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000007e-06 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378662+0000", 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8287000000000004e-05 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378672+0000", 2024-09-15T01:33:12.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3840000000000003e-06 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378705+0000", 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3506000000000001e-05 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378722+0000", 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6449000000000001e-05 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378843+0000", 2024-09-15T01:33:12.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000120937 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=82, reqid=client.4477.0:281, at_version=44'36, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.243124+0000", 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.887181188, 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013792832999999999, 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243124+0000", 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243124+0000", 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243125+0000", 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4299999999999999e-07 2024-09-15T01:33:12.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243145+0000", 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0360000000000002e-05 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243168+0000", 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2799e-05 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243173+0000", 2024-09-15T01:33:12.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5999999999999997e-06 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243212+0000", 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8565000000000001e-05 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243221+0000", 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6190000000000006e-06 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243416+0000", 2024-09-15T01:33:12.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019436 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243422+0000", 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6540000000000004e-06 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244114+0000", 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00069206699999999999 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244150+0000", 2024-09-15T01:33:12.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6408000000000001e-05 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256917+0000", 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012767058 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=83, reqid=osd.11.0:83, at_version=44'38, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.245647+0000", 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.884658701999999, 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013964298999999999, 2024-09-15T01:33:12.156 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245647+0000", 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245647+0000", 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245647+0000", 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245666+0000", 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9114e-05 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245699+0000", 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2186e-05 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245705+0000", 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4130000000000003e-06 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245749+0000", 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4465999999999997e-05 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245759+0000", 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8279999999999998e-06 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245944+0000", 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000184647 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245950+0000", 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0789999999999997e-06 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245981+0000", 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0842e-05 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7070000000000004e-06 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259611+0000", 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013620423 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.247204+0000", 2024-09-15T01:33:12.161 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.883100897999999, 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017667424000000001, 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247204+0000", 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247204+0000", 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247205+0000", 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247219+0000", 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4110999999999999e-05 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247242+0000", 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3064000000000001e-05 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247248+0000", 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6470000000000003e-06 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247285+0000", 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7526e-05 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247292+0000", 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4760000000000003e-06 2024-09-15T01:33:12.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250915+0000", 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036228549999999999 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250922+0000", 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9199999999999998e-06 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250953+0000", 2024-09-15T01:33:12.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1126e-05 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251618+0000", 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00066501799999999997 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264872+0000", 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013254203000000001 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 44/31 ECSubWrite(tid=28, reqid=osd.8.0:28, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.250047+0000", 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.880258504, 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014788809, 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250047+0000", 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250047+0000", 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250047+0000", 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.09e-07 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250060+0000", 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.257e-05 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250086+0000", 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6308e-05 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250092+0000", 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0290000000000002e-06 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250142+0000", 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9651999999999998e-05 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250152+0000", 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0098000000000001e-05 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251212+0000", 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001059854 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251221+0000", 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9970000000000004e-06 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251243+0000", 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2948000000000001e-05 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251256+0000", 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2743e-05 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264836+0000", 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013579501000000001 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.251636+0000", 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.878669041999999, 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024080965999999999, 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251636+0000", 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251636+0000", 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251637+0000", 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7800000000000001e-07 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251655+0000", 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8062000000000001e-05 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251686+0000", 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0723999999999999e-05 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251690+0000", 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0049999999999998e-06 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251718+0000", 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8486000000000001e-05 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251725+0000", 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.252412+0000", 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068728599999999995 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.252417+0000", 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.253911+0000", 2024-09-15T01:33:12.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001493858 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.253956+0000", 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4316000000000001e-05 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275717+0000", 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021761656000000001 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:282 3.3s0 3:c4c60c95:::smithi17529310-28:head [delete] snapc 8=[8] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.256127+0000", 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.874178783000001, 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019700597, 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 282 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256127+0000", 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256127+0000", 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256128+0000", 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3030000000000001e-06 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256132+0000", 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9990000000000002e-06 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256143+0000", 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1375e-05 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256147+0000", 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1949999999999996e-06 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256180+0000", 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2387e-05 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256244+0000", 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4369000000000007e-05 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256731+0000", 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00048626099999999998 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256921+0000", 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000190743 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256928+0000", 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8510000000000001e-06 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256963+0000", 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5314000000000001e-05 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256979+0000", 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5174000000000001e-05 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.258442+0000", 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00146372 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275827+0000", 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017384905999999999 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=85, reqid=osd.11.0:84, at_version=0'0, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.257015+0000", 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.873290889, 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019035946000000002, 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257015+0000", 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257015+0000", 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257015+0000", 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7100000000000002e-07 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257025+0000", 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0115999999999999e-05 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257047+0000", 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1645e-05 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257051+0000", 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4139999999999996e-06 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257094+0000", 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2843999999999998e-05 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257104+0000", 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.798e-06 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257241+0000", 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013668799999999999 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257248+0000", 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5299999999999999e-06 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259196+0000", 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019477850000000001 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259787+0000", 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00059073299999999997 2024-09-15T01:33:12.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.276050+0000", 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016263822000000001 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 44/31 ECSubWrite(tid=30, reqid=osd.8.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.257430+0000", 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.872874928999998, 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018688996999999999, 2024-09-15T01:33:12.188 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257430+0000", 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257430+0000", 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257431+0000", 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2499999999999995e-07 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257443+0000", 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1728e-05 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257465+0000", 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2013000000000001e-05 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257469+0000", 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6750000000000001e-06 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257498+0000", 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8670000000000002e-05 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257503+0000", 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1019999999999996e-06 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257572+0000", 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8450999999999999e-05 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259198+0000", 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001626239 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259263+0000", 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5491e-05 2024-09-15T01:33:12.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259288+0000", 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4766e-05 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.276119+0000", 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016831336999999998 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=86, reqid=osd.11.0:86, at_version=44'42, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.257959+0000", 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.872346615000001, 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018797019000000002, 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257959+0000", 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257959+0000", 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257959+0000", 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257972+0000", 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.261e-05 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257993+0000", 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1067e-05 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257996+0000", 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4690000000000002e-06 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.258029+0000", 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2691999999999998e-05 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.258035+0000", 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0619999999999997e-06 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259524+0000", 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001488781 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259533+0000", 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3190000000000001e-06 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259567+0000", 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3856999999999998e-05 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259896+0000", 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032855100000000001 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.276756+0000", 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016860199999999999 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.198 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.259034+0000", 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.871271657000001, 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018774057, 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259034+0000", 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259034+0000", 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259036+0000", 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0829999999999998e-06 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259049+0000", 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3283e-05 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259070+0000", 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0488000000000001e-05 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259073+0000", 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5310000000000002e-06 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259569+0000", 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049599699999999997 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259578+0000", 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7709999999999995e-06 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259766+0000", 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000187954 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259774+0000", 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1920000000000005e-06 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259800+0000", 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.635e-05 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259910+0000", 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109957 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.277808+0000", 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017897451000000002 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 44/31 ECSubWrite(tid=31, reqid=client.4477.0:288, at_version=44'22, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.264059+0000", 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.866246024999999, 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015322966, 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264059+0000", 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264059+0000", 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264061+0000", 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3829999999999999e-06 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264453+0000", 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039225000000000002 2024-09-15T01:33:12.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264486+0000", 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3380999999999998e-05 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264493+0000", 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3230000000000003e-06 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264530+0000", 2024-09-15T01:33:12.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7138000000000003e-05 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264540+0000", 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8409999999999995e-06 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264775+0000", 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023527500000000001 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264785+0000", 2024-09-15T01:33:12.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0107e-05 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264904+0000", 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119039 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264924+0000", 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9732999999999999e-05 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.279382+0000", 2024-09-15T01:33:12.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014458496 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 47/31 ECSubWriteReply(tid=162, last_complete=47'9, committed=1, applied=1))", 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:49.375678+0000", 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.75462783, 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.71735708600000003, 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.375678+0000", 2024-09-15T01:33:12.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.375678+0000", 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.375678+0000", 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2800000000000003e-07 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.375680+0000", 2024-09-15T01:33:12.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.635e-06 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.092521+0000", 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.716841324 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.092538+0000", 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6983e-05 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.092633+0000", 2024-09-15T01:33:12.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5618999999999994e-05 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.093035+0000", 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000401197 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:397 3.2s0 3:44cf4bb4:::smithi17529310-34:head [append 0~770048] snapc e=[e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:50.276037+0000", 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.854268799, 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013534935999999999, 2024-09-15T01:33:12.212 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 397 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.276037+0000", 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.276037+0000", 2024-09-15T01:33:12.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.276039+0000", 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9089999999999998e-06 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.282460+0000", 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0064216500000000001 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.282480+0000", 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0267000000000001e-05 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.282488+0000", 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2729999999999997e-06 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.282537+0000", 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9740999999999997e-05 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.282615+0000", 2024-09-15T01:33:12.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7773000000000002e-05 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.283687+0000", 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010715869999999999 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.283942+0000", 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025488699999999997 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.283955+0000", 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3328e-05 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.284073+0000", 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117716 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.284088+0000", 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5247e-05 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.289497+0000", 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005408698 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.289572+0000", 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4859999999999998e-05 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:509 3.2s0 3:4568dce4:::smithi17529310-45:head [copy-from ver 70] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:12.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.073110+0000", 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.057195233, 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.92744239399999995, 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 509 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073110+0000", 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073110+0000", 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073112+0000", 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073120+0000", 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8269999999999998e-06 2024-09-15T01:33:12.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073134+0000", 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3934000000000001e-05 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073142+0000", 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9899999999999997e-06 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073519+0000", 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037723500000000002 2024-09-15T01:33:12.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073522+0000", 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9179999999999998e-06 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.985732+0000", 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.91221027799999999 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.985887+0000", 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015458799999999999 2024-09-15T01:33:12.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.997489+0000", 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011601800000000001 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.997763+0000", 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000274586 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.997775+0000", 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1488000000000001e-05 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.998089+0000", 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031444300000000003 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.998169+0000", 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9367000000000001e-05 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.000501+0000", 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023319030000000002 2024-09-15T01:33:12.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.000553+0000", 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1966999999999999e-05 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:510 3.2s0 3:4568dce4:::smithi17529310-45:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.073150+0000", 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.057155845, 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.92501677100000002, 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.225 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 510 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073150+0000", 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073150+0000", 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073150+0000", 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6300000000000005e-07 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073157+0000", 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5490000000000003e-06 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073165+0000", 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.926e-06 2024-09-15T01:33:12.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073169+0000", 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4429999999999996e-06 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073547+0000", 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037755599999999997 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-15T01:33:12.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073549+0000", 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.616e-06 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.986765+0000", 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.91321570200000002 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.986800+0000", 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5169999999999997e-05 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.997953+0000", 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011152998000000001 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.998084+0000", 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013044400000000001 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.998166+0000", 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2804000000000002e-05 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:520 3.2s0 3:44cf4bb4:::smithi17529310-34:head [stat,delete] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:58.071228+0000", 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.059077125, 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016403424, 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 520 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.231 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.071228+0000", 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.071228+0000", 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.071230+0000", 2024-09-15T01:33:12.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.739e-06 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.071236+0000", 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8130000000000001e-06 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.071249+0000", 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3117e-05 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.071256+0000", 2024-09-15T01:33:12.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5449999999999998e-06 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.072957+0000", 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017000870000000001 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.073049+0000", 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2130000000000001e-05 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.073899+0000", 2024-09-15T01:33:12.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084990499999999997 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.076533+0000", 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026339330000000002 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.078961+0000", 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00242845 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.079063+0000", 2024-09-15T01:33:12.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010181199999999999 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.079342+0000", 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027918499999999999 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.079396+0000", 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4490999999999997e-05 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.087632+0000", 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082352169999999995 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 63/31 ECSubWrite(tid=46, reqid=client.4477.0:596, at_version=0'0, trim_to=0'0, roll_forward_to=63'33))", 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:05.101346+0000", 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.0289594480000002, 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.96745142500000003, 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:12.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:05.101346+0000", 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:05.101346+0000", 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:05.101347+0000", 2024-09-15T01:33:12.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:05.101358+0000", 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0951000000000001e-05 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068502+0000", 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.96714420499999998 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068510+0000", 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1589999999999993e-06 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068576+0000", 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5767999999999998e-05 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068586+0000", 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-06 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068671+0000", 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5958000000000006e-05 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068684+0000", 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2228999999999999e-05 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068724+0000", 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0299999999999997e-05 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068738+0000", 2024-09-15T01:33:12.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3711e-05 2024-09-15T01:33:12.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:12.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:12.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:12.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068797+0000", 2024-09-15T01:33:12.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9648000000000002e-05 2024-09-15T01:33:12.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.242 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:12.242 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:12.242 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:12.244 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:33:12.244 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:12.242+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:12.344 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:12.342+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:12.352 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:12.353 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:12.361 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:33:12.445 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:12.443+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:12.459 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:12.459 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:12.459 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:12.459 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:12.460 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:12.468 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:33:12.545 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:12.544+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:12.567 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:12.567 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:12.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:12.567 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=152, reqid=osd.13.0:152, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.243665+0000", 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.320920003000001, 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019478882999999999, 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243665+0000", 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243665+0000", 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243666+0000", 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.34e-07 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243686+0000", 2024-09-15T01:33:12.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9830000000000002e-05 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243778+0000", 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2367999999999994e-05 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243802+0000", 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3646e-05 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243862+0000", 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0579999999999999e-05 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243876+0000", 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4025e-05 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245801+0000", 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0019244399999999999 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246115+0000", 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00031487699999999997 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248315+0000", 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021992140000000001 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248329+0000", 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4450000000000001e-05 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263144+0000", 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014815218999999999 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.317617401, 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019581815999999998, 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246969+0000", 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7000000000000003e-07 2024-09-15T01:33:12.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247008+0000", 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9354999999999998e-05 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247051+0000", 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2849000000000003e-05 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247060+0000", 2024-09-15T01:33:12.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4589999999999998e-06 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248680+0000", 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001620058 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248690+0000", 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8660000000000001e-06 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248945+0000", 2024-09-15T01:33:12.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025509900000000001 2024-09-15T01:33:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251105+0000", 2024-09-15T01:33:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021604290000000002 2024-09-15T01:33:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251177+0000", 2024-09-15T01:33:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1680999999999994e-05 2024-09-15T01:33:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251189+0000", 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2661e-05 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266550+0000", 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015360389 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=28, reqid=osd.8.0:28, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.314435530000001, 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022199446000000001, 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4700000000000001e-07 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250167+0000", 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7384000000000002e-05 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250202+0000", 2024-09-15T01:33:12.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4912e-05 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250210+0000", 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6029999999999999e-06 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250258+0000", 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8109999999999998e-05 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250265+0000", 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4400000000000002e-06 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250466+0000", 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000201227 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251113+0000", 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00064769299999999997 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251179+0000", 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5895999999999994e-05 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251196+0000", 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6810999999999999e-05 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272349+0000", 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021153222999999999 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.583 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.313081402000002, 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023186075, 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:33:12.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251505+0000", 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.09e-07 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251531+0000", 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5834999999999999e-05 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251574+0000", 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2879000000000001e-05 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251580+0000", 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8460000000000002e-06 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251639+0000", 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8331000000000001e-05 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251649+0000", 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9930000000000007e-06 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251890+0000", 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00024173400000000001 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251923+0000", 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2607999999999997e-05 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251985+0000", 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2238999999999998e-05 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251998+0000", 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2979999999999999e-05 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274690+0000", 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022691821000000001 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s1 44/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=44'26))", 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.311864404000001, 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019696631999999999, 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252722+0000", 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3099999999999996e-07 2024-09-15T01:33:12.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252734+0000", 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1576999999999999e-05 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252780+0000", 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6820999999999998e-05 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252788+0000", 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2710000000000003e-06 2024-09-15T01:33:12.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252848+0000", 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0718999999999999e-05 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252859+0000", 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0342e-05 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252960+0000", 2024-09-15T01:33:12.592 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000101741 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252970+0000", 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0008e-05 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253967+0000", 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00099612599999999996 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253987+0000", 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0064e-05 2024-09-15T01:33:12.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018431032 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=154, reqid=client.4477.0:282, at_version=44'19, trim_to=0'0, roll_forward_to=44'17))", 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.307852744000002, 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023652260000000001, 2024-09-15T01:33:12.594 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256734+0000", 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3900000000000004e-07 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256754+0000", 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0279e-05 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256785+0000", 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0703000000000001e-05 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.596 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256798+0000", 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3606e-05 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256843+0000", 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5065000000000003e-05 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256851+0000", 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5580000000000003e-06 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.597 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257013+0000", 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016264 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257339+0000", 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032564200000000001 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257402+0000", 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3162000000000003e-05 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.598 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258083+0000", 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00068126099999999995 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280385+0000", 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022301504999999999 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=85, reqid=osd.11.0:84, at_version=0'0, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:33:12.599 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.307528959999999, 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020078747000000001, 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257057+0000", 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5000000000000002e-07 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257067+0000", 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5540000000000003e-06 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257097+0000", 2024-09-15T01:33:12.601 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9892000000000001e-05 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257102+0000", 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3360000000000002e-06 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257155+0000", 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3313000000000003e-05 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257164+0000", 2024-09-15T01:33:12.602 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1060000000000006e-06 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257257+0000", 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2678999999999998e-05 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257341+0000", 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3459000000000002e-05 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257403+0000", 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2477000000000005e-05 2024-09-15T01:33:12.603 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257420+0000", 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7170999999999999e-05 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277135+0000", 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019715010000000002 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.604 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=30, reqid=osd.8.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.307261689000001, 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020096306000000001, 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.605 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3099999999999997e-07 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257335+0000", 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0727000000000001e-05 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.606 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257371+0000", 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5583999999999997e-05 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257378+0000", 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.754e-06 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257427+0000", 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9048999999999997e-05 2024-09-15T01:33:12.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257475+0000", 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8140999999999999e-05 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257605+0000", 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012981700000000001 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257640+0000", 2024-09-15T01:33:12.608 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.557e-05 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257674+0000", 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4215999999999999e-05 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257692+0000", 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7988000000000001e-05 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.609 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277420+0000", 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019727728999999999 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=86, reqid=osd.11.0:86, at_version=44'42, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.306670435000001, 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022807320999999998, 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.610 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257916+0000", 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:33:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257951+0000", 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5754999999999999e-05 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257982+0000", 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0732000000000003e-05 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.612 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257987+0000", 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1900000000000003e-06 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258036+0000", 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8547999999999997e-05 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258047+0000", 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1082e-05 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.613 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258512+0000", 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00046539599999999997 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258527+0000", 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5121e-05 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258568+0000", 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0305000000000001e-05 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258581+0000", 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3434999999999999e-05 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280722+0000", 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022141137000000002 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=155, reqid=client.4477.0:282, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.305745638000001, 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021774615000000001, 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.616 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258841+0000", 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5899999999999995e-07 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258853+0000", 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2115e-05 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.617 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258889+0000", 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6007000000000003e-05 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258896+0000", 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258958+0000", 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2242999999999993e-05 2024-09-15T01:33:12.618 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258966+0000", 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2370000000000003e-06 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259070+0000", 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000104171 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259104+0000", 2024-09-15T01:33:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4569000000000001e-05 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259140+0000", 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5698000000000003e-05 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259155+0000", 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4541e-05 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280614+0000", 2024-09-15T01:33:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021459865000000002 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.305316263000002, 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027513342999999999, 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259270+0000", 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.33e-07 2024-09-15T01:33:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259295+0000", 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4413000000000002e-05 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259335+0000", 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0154999999999998e-05 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259344+0000", 2024-09-15T01:33:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8890000000000008e-06 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259414+0000", 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9929000000000001e-05 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259423+0000", 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0289999999999994e-06 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263376+0000", 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003953036 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263393+0000", 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7444000000000001e-05 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263481+0000", 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8251999999999995e-05 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266784+0000", 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003303074 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286783+0000", 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019998189 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=31, reqid=client.4477.0:288, at_version=44'22, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.300548891999998, 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022720738000000001, 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6599999999999995e-07 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264398+0000", 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00036057299999999999 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264436+0000", 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7968e-05 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264444+0000", 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8949999999999998e-06 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264489+0000", 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4954999999999999e-05 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264499+0000", 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0574000000000001e-05 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266178+0000", 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001678543 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266188+0000", 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0624e-05 2024-09-15T01:33:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268728+0000", 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025397240000000001 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268912+0000", 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018415400000000001 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286757+0000", 2024-09-15T01:33:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.017844961999999999 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=32, reqid=client.4477.0:289, at_version=44'23, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.297349453999999, 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019719520000000001, 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:33:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:33:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267237+0000", 2024-09-15T01:33:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3099999999999998e-07 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267495+0000", 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025857700000000001 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267528+0000", 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2617999999999998e-05 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267534+0000", 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0159999999999997e-06 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267584+0000", 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0176999999999997e-05 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267594+0000", 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0161e-05 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267768+0000", 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017420000000000001 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268732+0000", 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00096413399999999995 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268818+0000", 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5583999999999999e-05 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268943+0000", 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012452399999999999 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286956+0000", 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018012997999999999 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=91, reqid=osd.11.0:87, at_version=0'0, trim_to=0'0, roll_forward_to=44'44))", 2024-09-15T01:33:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.297257009999999, 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019815672999999999, 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267329+0000", 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7500000000000001e-07 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267339+0000", 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0351e-05 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267385+0000", 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6008999999999999e-05 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267393+0000", 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7810000000000005e-06 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267443+0000", 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0139000000000002e-05 2024-09-15T01:33:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267454+0000", 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1012999999999999e-05 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267585+0000", 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013078699999999999 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268732+0000", 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001146768 2024-09-15T01:33:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268817+0000", 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5724999999999997e-05 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268838+0000", 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0954e-05 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.287144+0000", 2024-09-15T01:33:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018305770999999998 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=32, reqid=client.4477.0:294, at_version=44'23, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.294282797000001, 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021391184000000001, 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270304+0000", 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7499999999999998e-07 2024-09-15T01:33:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270326+0000", 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2821999999999999e-05 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270366+0000", 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9708e-05 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270374+0000", 2024-09-15T01:33:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6709999999999999e-06 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270433+0000", 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9274999999999997e-05 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270444+0000", 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.131e-05 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271974+0000", 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001529996 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271986+0000", 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1443e-05 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272180+0000", 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019405299999999999 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272366+0000", 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018627699999999999 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.291694+0000", 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019327654 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.293686119, 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024535670999999998, 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:33:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270900+0000", 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.23e-07 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270920+0000", 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.022e-05 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270953+0000", 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3136000000000003e-05 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270960+0000", 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4280000000000001e-06 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272041+0000", 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010816440000000001 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272050+0000", 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5269999999999998e-06 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272149+0000", 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9464999999999997e-05 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272182+0000", 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2792e-05 2024-09-15T01:33:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272209+0000", 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6409000000000001e-05 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272435+0000", 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000226691 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295435+0000", 2024-09-15T01:33:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022999636 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=34, reqid=client.4477.0:296, at_version=44'25, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.293120011999999, 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024251137999999998, 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271466+0000", 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8899999999999998e-07 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271488+0000", 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1875999999999999e-05 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271522+0000", 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4041999999999997e-05 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271529+0000", 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4620000000000001e-06 2024-09-15T01:33:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272189+0000", 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066068699999999997 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272198+0000", 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2900000000000002e-06 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272284+0000", 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6638999999999995e-05 2024-09-15T01:33:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272322+0000", 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7125000000000001e-05 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272357+0000", 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5626000000000001e-05 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272458+0000", 2024-09-15T01:33:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000100343 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295717+0000", 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023259058999999999 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=35, reqid=client.4477.0:291, at_version=0'0, trim_to=0'0, roll_forward_to=44'25))", 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.291512874999999, 2024-09-15T01:33:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022393413000000001, 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0999999999999998e-07 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273083+0000", 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0015e-05 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273110+0000", 2024-09-15T01:33:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6798999999999999e-05 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273117+0000", 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5379999999999999e-06 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273155+0000", 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8563999999999999e-05 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273163+0000", 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4939999999999997e-06 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273248+0000", 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5290000000000005e-05 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273254+0000", 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3149999999999997e-06 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273288+0000", 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3958000000000003e-05 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273303+0000", 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4731e-05 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295466+0000", 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022162998999999999 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 54/31 ECSubWrite(tid=63, reqid=client.4477.0:459, at_version=54'49, trim_to=0'0, roll_forward_to=54'46))", 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.094073+0000", 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.470511985000002, 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.82711023299999997, 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094073+0000", 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094073+0000", 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094074+0000", 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4099999999999999e-07 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094105+0000", 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1087000000000002e-05 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920807+0000", 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.82670179300000002 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920813+0000", 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8100000000000003e-06 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920878+0000", 2024-09-15T01:33:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4870999999999996e-05 2024-09-15T01:33:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920885+0000", 2024-09-15T01:33:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3860000000000001e-06 2024-09-15T01:33:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920975+0000", 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9797999999999996e-05 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920981+0000", 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2419999999999997e-06 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.921033+0000", 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1811999999999998e-05 2024-09-15T01:33:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.921048+0000", 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5135e-05 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.921184+0000", 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013575800000000001 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpRead(3.fs1 69/31 ECSubRead(tid=220, to_read={3:f73c1a72:::smithi17529310-29:head=0,4096,0}, subchunks={3:f73c1a72:::smithi17529310-29:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:11.104731+0000", 2024-09-15T01:33:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 1.4598547230000001, 2024-09-15T01:33:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.77828088100000004, 2024-09-15T01:33:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.104731+0000", 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.104731+0000", 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.104731+0000", 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8699999999999999e-07 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.104736+0000", 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7659999999999998e-06 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.882809+0000", 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.77807284600000004 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.882814+0000", 2024-09-15T01:33:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8889999999999999e-06 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.882901+0000", 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6193000000000006e-05 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.883012+0000", 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000111 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:12.675 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:12.676 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:33:12.677 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:12.674+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:12.774 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:12.774 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:12.775 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:12.775 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:12.777 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:12.775+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:12.783 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:33:12.877 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:12.876+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:12.880 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:12.880 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:12.880 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:12.880 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:12.888 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:33:12.978 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:12.976+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:13.027 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:13.027 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:13.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:13.027 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:13.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.028 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:13.028 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:13.028 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.78375248, 2024-09-15T01:33:13.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022664864999999999, 2024-09-15T01:33:13.028 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.028 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.028 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:13.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:13.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:13.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.16e-07 2024-09-15T01:33:13.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241155+0000", 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016715400000000001 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241334+0000", 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017883600000000001 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241342+0000", 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1519999999999992e-06 2024-09-15T01:33:13.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241446+0000", 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010419 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241468+0000", 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1747e-05 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.244959+0000", 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0034911600000000001 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.244970+0000", 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1131000000000001e-05 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245013+0000", 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3214000000000003e-05 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245498+0000", 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00048501499999999999 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263652+0000", 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018154050000000001 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.783554519999999, 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022656372000000001, 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9399999999999999e-07 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241203+0000", 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7152000000000001e-05 2024-09-15T01:33:13.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241707+0000", 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050421299999999999 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241714+0000", 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4409999999999998e-06 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241773+0000", 2024-09-15T01:33:13.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9762000000000001e-05 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241783+0000", 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6860000000000001e-06 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245035+0000", 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032521130000000001 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245184+0000", 2024-09-15T01:33:13.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014925800000000001 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245769+0000", 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058491399999999996 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246140+0000", 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037060199999999999 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263842+0000", 2024-09-15T01:33:13.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.017702037 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.782337402, 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034719353000000001, 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1600000000000001e-07 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242427+0000", 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4022000000000001e-05 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242460+0000", 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2496000000000002e-05 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242476+0000", 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6503000000000001e-05 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245083+0000", 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.002606531 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245092+0000", 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7679999999999997e-06 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245298+0000", 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000205971 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245772+0000", 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00047452200000000001 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245817+0000", 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4474999999999997e-05 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246171+0000", 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035433200000000001 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277122+0000", 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030951117 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.781713871000001, 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031640535999999997, 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.044 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.49e-07 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243044+0000", 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7946999999999999e-05 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243140+0000", 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5311999999999998e-05 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243147+0000", 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2069999999999998e-06 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245010+0000", 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018629230000000001 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245020+0000", 2024-09-15T01:33:13.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0774999999999999e-05 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245472+0000", 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045200900000000002 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245773+0000", 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00030029999999999998 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245817+0000", 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4367999999999999e-05 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246317+0000", 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050021500000000003 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274667+0000", 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028349330999999998 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.049 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.777449825000001, 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027771575999999999, 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247291+0000", 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7700000000000004e-07 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247302+0000", 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1246e-05 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247333+0000", 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1179000000000001e-05 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247341+0000", 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.571e-06 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247388+0000", 2024-09-15T01:33:13.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6876999999999999e-05 2024-09-15T01:33:13.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247395+0000", 2024-09-15T01:33:13.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3359999999999997e-06 2024-09-15T01:33:13.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247478+0000", 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2860000000000003e-05 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247487+0000", 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9029999999999993e-06 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247526+0000", 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8829999999999999e-05 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247546+0000", 2024-09-15T01:33:13.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0551999999999999e-05 2024-09-15T01:33:13.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275062+0000", 2024-09-15T01:33:13.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027515444999999999 2024-09-15T01:33:13.055 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.055 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.055 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as5 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.777227005, 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027915809999999999, 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247514+0000", 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9899999999999999e-07 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247524+0000", 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0396e-05 2024-09-15T01:33:13.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247555+0000", 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0414000000000001e-05 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247564+0000", 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1900000000000001e-06 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247616+0000", 2024-09-15T01:33:13.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1788e-05 2024-09-15T01:33:13.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247628+0000", 2024-09-15T01:33:13.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.288e-05 2024-09-15T01:33:13.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247763+0000", 2024-09-15T01:33:13.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013473700000000001 2024-09-15T01:33:13.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247772+0000", 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3429999999999996e-06 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247804+0000", 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2533999999999997e-05 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247817+0000", 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2694000000000001e-05 2024-09-15T01:33:13.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275429+0000", 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027612035 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s5 44/31 ECSubWrite(tid=65, reqid=osd.14.0:65, at_version=44'26, trim_to=0'0, roll_forward_to=44'24))", 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.775181659000001, 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.026123550999999998, 2024-09-15T01:33:13.061 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249559+0000", 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3200000000000003e-07 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249584+0000", 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4284999999999999e-05 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249633+0000", 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9526e-05 2024-09-15T01:33:13.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249641+0000", 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2239999999999995e-06 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249695+0000", 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3674999999999997e-05 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249705+0000", 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8300000000000008e-06 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251518+0000", 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018128409999999999 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251549+0000", 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0948999999999998e-05 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251591+0000", 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2515999999999997e-05 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251612+0000", 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0494000000000001e-05 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275682+0000", 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024070279 2024-09-15T01:33:13.066 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s5 44/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=44'26))", 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.771987949, 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022959355000000001, 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:13.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:13.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252753+0000", 2024-09-15T01:33:13.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7499999999999998e-07 2024-09-15T01:33:13.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252764+0000", 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1123e-05 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252899+0000", 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013444100000000001 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252906+0000", 2024-09-15T01:33:13.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8650000000000001e-06 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252959+0000", 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2698000000000003e-05 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252968+0000", 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2140000000000002e-06 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253058+0000", 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9926999999999995e-05 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253066+0000", 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6620000000000005e-06 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253105+0000", 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8875999999999998e-05 2024-09-15T01:33:13.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253117+0000", 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2405000000000001e-05 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275711+0000", 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022594169000000001 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=60, reqid=client.4477.0:306, at_version=44'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.753911549000001, 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024129333999999999, 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270829+0000", 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270845+0000", 2024-09-15T01:33:13.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6013999999999999e-05 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270871+0000", 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6140999999999999e-05 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270877+0000", 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7989999999999999e-06 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270947+0000", 2024-09-15T01:33:13.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9480000000000006e-05 2024-09-15T01:33:13.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270959+0000", 2024-09-15T01:33:13.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2395000000000001e-05 2024-09-15T01:33:13.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274484+0000", 2024-09-15T01:33:13.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003524976 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274715+0000", 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000230595 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274806+0000", 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1130000000000003e-05 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274847+0000", 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1473000000000001e-05 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.294958+0000", 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020110711 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:13.078 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.753833538999999, 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020359914999999999, 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270907+0000", 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4200000000000001e-07 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270919+0000", 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1761e-05 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270936+0000", 2024-09-15T01:33:13.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7761000000000002e-05 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270953+0000", 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6858e-05 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272042+0000", 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010882769999999999 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272051+0000", 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8400000000000007e-06 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272145+0000", 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3639000000000003e-05 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272153+0000", 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6240000000000002e-06 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272221+0000", 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8281999999999994e-05 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272241+0000", 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9542e-05 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.291266+0000", 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019025889000000001 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.083 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.752322292999999, 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.025858174000000001, 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272446+0000", 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7114000000000002e-05 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272489+0000", 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3711999999999998e-05 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272498+0000", 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9169999999999995e-06 2024-09-15T01:33:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274537+0000", 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0020384190000000001 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274545+0000", 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1999999999999994e-06 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274659+0000", 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011420499999999999 2024-09-15T01:33:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274716+0000", 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7114999999999998e-05 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274807+0000", 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0346999999999994e-05 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274883+0000", 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.695e-05 2024-09-15T01:33:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298276+0000", 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023392452000000001 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=62, reqid=client.4477.0:308, at_version=44'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.751275143000001, 2024-09-15T01:33:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024943603000000002, 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.37e-07 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273478+0000", 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2768e-05 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273502+0000", 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3920000000000001e-05 2024-09-15T01:33:13.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273508+0000", 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0909999999999998e-06 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274699+0000", 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011909240000000001 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274707+0000", 2024-09-15T01:33:13.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5429999999999996e-06 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274788+0000", 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1804000000000005e-05 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274811+0000", 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2379e-05 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274902+0000", 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1031000000000003e-05 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274910+0000", 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8359999999999992e-06 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298408+0000", 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023498769999999999 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.094 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=92, reqid=client.4477.0:309, at_version=44'31, trim_to=0'0, roll_forward_to=44'28))", 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.750175665, 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024021724000000001, 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:13.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274565+0000", 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9299999999999996e-07 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275016+0000", 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045075999999999997 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275077+0000", 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0582999999999999e-05 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275084+0000", 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1289999999999998e-06 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275144+0000", 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0566000000000003e-05 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275153+0000", 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4849999999999992e-06 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275302+0000", 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000149689 2024-09-15T01:33:13.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275327+0000", 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4337000000000001e-05 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277020+0000", 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001693153 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277074+0000", 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4122999999999997e-05 2024-09-15T01:33:13.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298586+0000", 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021512006 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 44/31 ECSubWrite(tid=35, reqid=client.4477.0:296, at_version=0'0, trim_to=0'0, roll_forward_to=44'25))", 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.746633691, 2024-09-15T01:33:13.100 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020797596000000002, 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278107+0000", 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8199999999999998e-07 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278119+0000", 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1567e-05 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278155+0000", 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6560999999999998e-05 2024-09-15T01:33:13.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278163+0000", 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2819999999999999e-06 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278209+0000", 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6146999999999997e-05 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278219+0000", 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0699999999999999e-05 2024-09-15T01:33:13.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278342+0000", 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000122061 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278350+0000", 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4460000000000001e-06 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280088+0000", 2024-09-15T01:33:13.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0017376309999999999 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280107+0000", 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.986e-05 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298904+0000", 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018796459000000001 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=93, reqid=client.4477.0:310, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.745925688, 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020226037999999998, 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.106 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278815+0000", 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6000000000000003e-07 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279204+0000", 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000388413 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279228+0000", 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4674000000000001e-05 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279233+0000", 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7849999999999999e-06 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279272+0000", 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9502999999999998e-05 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279278+0000", 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0079999999999999e-06 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279401+0000", 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012238799999999999 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280091+0000", 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00069018900000000002 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280213+0000", 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012227700000000001 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280231+0000", 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7408999999999999e-05 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.299040+0000", 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018809632 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.110 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.5s6 44/31 ECSubWrite(tid=15, reqid=client.4477.0:314, at_version=44'8, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.745376232000002, 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.026604415999999999, 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279386+0000", 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.211e-05 2024-09-15T01:33:13.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279421+0000", 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4523000000000001e-05 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279428+0000", 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4420000000000001e-06 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279474+0000", 2024-09-15T01:33:13.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5995000000000003e-05 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279480+0000", 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.105e-06 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279645+0000", 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016430199999999999 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280092+0000", 2024-09-15T01:33:13.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044707999999999998 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280214+0000", 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012248399999999999 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280260+0000", 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5785000000000003e-05 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.305968+0000", 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.025708106000000001 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.5s6 44/31 ECSubWrite(tid=16, reqid=client.4477.0:314, at_version=0'0, trim_to=0'0, roll_forward_to=44'8))", 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.742721658000001, 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024148095000000001, 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.116 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282019+0000", 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282028+0000", 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0969999999999995e-06 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282054+0000", 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5897e-05 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282059+0000", 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7849999999999999e-06 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282101+0000", 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1801000000000002e-05 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282108+0000", 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0650000000000001e-06 2024-09-15T01:33:13.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282171+0000", 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2834000000000002e-05 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282844+0000", 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00067343600000000002 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282915+0000", 2024-09-15T01:33:13.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1242999999999995e-05 2024-09-15T01:33:13.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282933+0000", 2024-09-15T01:33:13.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8136000000000001e-05 2024-09-15T01:33:13.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306166+0000", 2024-09-15T01:33:13.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023233052000000001 2024-09-15T01:33:13.121 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=63, reqid=client.4477.0:308, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 26.741138026000002, 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022584495, 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283603+0000", 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5899999999999995e-07 2024-09-15T01:33:13.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283618+0000", 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5600999999999999e-05 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283648+0000", 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9295e-05 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283653+0000", 2024-09-15T01:33:13.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1050000000000003e-06 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283703+0000", 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9883999999999999e-05 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283712+0000", 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3740000000000004e-06 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283859+0000", 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000146959 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283869+0000", 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0295999999999999e-05 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283906+0000", 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6534e-05 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283922+0000", 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6560000000000001e-05 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306187+0000", 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022264128000000001 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 47/38 ECSubWrite(tid=76, reqid=client.4477.0:376, at_version=47'25, trim_to=0'0, roll_forward_to=47'22))", 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 23.660933863, 2024-09-15T01:33:13.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.61852187999999997, 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363807+0000", 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5799999999999999e-07 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363818+0000", 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1729000000000001e-05 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981818+0000", 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.61799953399999996 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981826+0000", 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1049999999999995e-06 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981893+0000", 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7229000000000003e-05 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981905+0000", 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1527e-05 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982064+0000", 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000159147 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982078+0000", 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982122+0000", 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4354000000000003e-05 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982140+0000", 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8315e-05 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982328+0000", 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018781200000000001 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.132 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 54/52 ECSubWrite(tid=119, reqid=client.4477.0:500, at_version=54'41, trim_to=0'0, roll_forward_to=54'38))", 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.813438831999999, 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.27782579699999999, 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.133 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211302+0000", 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.89e-07 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211313+0000", 2024-09-15T01:33:13.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1521e-05 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488807+0000", 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.27749379200000002 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488811+0000", 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2889999999999999e-06 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.135 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488867+0000", 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5322000000000003e-05 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488874+0000", 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9659999999999996e-06 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488953+0000", 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8591999999999996e-05 2024-09-15T01:33:13.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488959+0000", 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0000000000000002e-06 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488993+0000", 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4326999999999998e-05 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.489009+0000", 2024-09-15T01:33:13.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6015000000000002e-05 2024-09-15T01:33:13.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.489127+0000", 2024-09-15T01:33:13.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117584 2024-09-15T01:33:13.138 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.138 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.138 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.138 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.138 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.138 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:13.139 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 9] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-15T01:33:13.139 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:33:13.139 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-15T01:33:13.139 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T01:33:13.141 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:13.137+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:252: done (5 left) 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:253: copy_from oid 4 from oid 18 current snap is 20 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 31 v 53 (ObjNum 85 snap 7 seq_num 85) dirty exists 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:244: expect (ObjNum 58 snap 3 seq_num 58) 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:246: finishing copy_from racing read to smithi17529310-6 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 12 v 67 (ObjNum 98 snap 10 seq_num 98) dirty exists 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:246: finishing copy_from to smithi17529310-6 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 6 v 36 (ObjNum 103 snap 10 seq_num 103) dirty exists 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:242: done (5 left) 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:244: done (4 left) 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:246: done (3 left) 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:250: done (2 left) 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:254: copy_from oid 3 from oid 29 current snap is 20 2024-09-15T01:33:13.142 INFO:tasks.rados.rados.0.smithi175.stdout:249: expect (ObjNum 114 snap 17 seq_num 114) 2024-09-15T01:33:13.143 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-15T01:33:13.143 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:33:13.204 INFO:tasks.rados.rados.0.smithi175.stdout:253: finishing copy_from racing read to smithi17529310-4 2024-09-15T01:33:13.267 INFO:tasks.rados.rados.0.smithi175.stdout:253: finishing copy_from to smithi17529310-4 2024-09-15T01:33:13.267 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 4 v 59 (ObjNum 115 snap 18 seq_num 115) dirty exists 2024-09-15T01:33:13.267 INFO:tasks.rados.rados.0.smithi175.stdout:254: finishing copy_from racing read to smithi17529310-3 2024-09-15T01:33:13.267 INFO:tasks.rados.rados.0.smithi175.stdout:254: finishing copy_from to smithi17529310-3 2024-09-15T01:33:13.267 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 3 v 121 (ObjNum 118 snap 19 seq_num 118) dirty exists 2024-09-15T01:33:13.267 INFO:tasks.rados.rados.0.smithi175.stdout:249: done (2 left) 2024-09-15T01:33:13.267 INFO:tasks.rados.rados.0.smithi175.stdout:253: done (1 left) 2024-09-15T01:33:13.267 INFO:tasks.rados.rados.0.smithi175.stdout:254: done (0 left) 2024-09-15T01:33:13.267 INFO:tasks.rados.rados.0.smithi175.stdout:255: rollback oid 29 current snap is 20 2024-09-15T01:33:13.267 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 29 to 17 2024-09-15T01:33:13.267 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 13 current snap is 20 2024-09-15T01:33:13.268 INFO:tasks.rados.rados.0.smithi175.stdout:256: seq_num 121 ranges {0=491520,491520=802816,1294336=720896,2015232=32768} 2024-09-15T01:33:13.272 INFO:tasks.rados.rados.0.smithi175.stdout:256: writing smithi17529310-13 from 0 to 491520 tid 1 2024-09-15T01:33:13.272 INFO:tasks.rados.rados.0.smithi175.stdout:256: writing smithi17529310-13 from 491520 to 1294336 tid 2 2024-09-15T01:33:13.272 INFO:tasks.rados.rados.0.smithi175.stdout:256: writing smithi17529310-13 from 1294336 to 2015232 tid 3 2024-09-15T01:33:13.272 INFO:tasks.rados.rados.0.smithi175.stdout:256: writing smithi17529310-13 from 2015232 to 2048000 tid 4 2024-09-15T01:33:13.272 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 34 current snap is 20 2024-09-15T01:33:13.272 INFO:tasks.rados.rados.0.smithi175.stdout:255: finishing rollback tid 0 to smithi17529310-29 2024-09-15T01:33:13.272 INFO:tasks.rados.rados.0.smithi175.stdout:255: finishing rollback tid 1 to smithi17529310-29 2024-09-15T01:33:13.272 INFO:tasks.rados.rados.0.smithi175.stdout:255: finishing rollback tid 2 to smithi17529310-29 2024-09-15T01:33:13.272 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 29 v 128 (ObjNum 68 snap 3 seq_num 68) dirty exists 2024-09-15T01:33:13.272 INFO:tasks.rados.rados.0.smithi175.stdout:256: finishing write tid 1 to smithi17529310-13 2024-09-15T01:33:13.272 INFO:tasks.rados.rados.0.smithi175.stdout:256: finishing write tid 2 to smithi17529310-13 2024-09-15T01:33:13.273 INFO:tasks.rados.rados.0.smithi175.stdout:257: seq_num 122 ranges {0=507904,507904=425984,933888=442368,1376256=540672,1916928=147456} 2024-09-15T01:33:13.273 INFO:tasks.rados.rados.0.smithi175.stdout:257: writing smithi17529310-34 from 0 to 507904 tid 1 2024-09-15T01:33:13.273 INFO:tasks.rados.rados.0.smithi175.stdout:257: writing smithi17529310-34 from 507904 to 933888 tid 2 2024-09-15T01:33:13.273 INFO:tasks.rados.rados.0.smithi175.stdout:257: writing smithi17529310-34 from 933888 to 1376256 tid 3 2024-09-15T01:33:13.273 INFO:tasks.rados.rados.0.smithi175.stdout:257: writing smithi17529310-34 from 1376256 to 1916928 tid 4 2024-09-15T01:33:13.273 INFO:tasks.rados.rados.0.smithi175.stdout:257: writing smithi17529310-34 from 1916928 to 2064384 tid 5 2024-09-15T01:33:13.273 INFO:tasks.rados.rados.0.smithi175.stdout:255: done (2 left) 2024-09-15T01:33:13.273 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 43 current snap is 20 2024-09-15T01:33:13.273 INFO:tasks.rados.rados.0.smithi175.stdout:256: finishing write tid 3 to smithi17529310-13 2024-09-15T01:33:13.273 INFO:tasks.rados.rados.0.smithi175.stdout:258: seq_num 123 ranges {1376256=524288,1900544=573440,2473984=688128,3162112=294912} 2024-09-15T01:33:13.273 INFO:tasks.rados.rados.0.smithi175.stdout:258: writing smithi17529310-43 from 1376256 to 1900544 tid 1 2024-09-15T01:33:13.273 INFO:tasks.rados.rados.0.smithi175.stdout:258: writing smithi17529310-43 from 1900544 to 2473984 tid 2 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:258: writing smithi17529310-43 from 2473984 to 3162112 tid 3 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:258: writing smithi17529310-43 from 3162112 to 3457024 tid 4 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 4 current snap is 20 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:256: finishing write tid 4 to smithi17529310-13 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:256: finishing write tid 5 to smithi17529310-13 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:257: finishing write tid 1 to smithi17529310-34 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:257: finishing write tid 2 to smithi17529310-34 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:257: finishing write tid 3 to smithi17529310-34 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:256: finishing write tid 6 to smithi17529310-13 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 13 v 71 (ObjNum 121 snap 20 seq_num 121) dirty exists 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:256: left oid 13 (ObjNum 121 snap 20 seq_num 121) 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:259: seq_num 124 ranges {2228224=540672,2768896=720896,3489792=786432,4276224=49152} 2024-09-15T01:33:13.274 INFO:tasks.rados.rados.0.smithi175.stdout:259: writing smithi17529310-4 from 2228224 to 2768896 tid 1 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:259: writing smithi17529310-4 from 2768896 to 3489792 tid 2 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:259: writing smithi17529310-4 from 3489792 to 4276224 tid 3 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:259: writing smithi17529310-4 from 4276224 to 4325376 tid 4 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:256: done (3 left) 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:260: delete oid 1 current snap is 20 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:257: finishing write tid 4 to smithi17529310-34 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:257: finishing write tid 5 to smithi17529310-34 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:257: finishing write tid 6 to smithi17529310-34 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:257: finishing write tid 7 to smithi17529310-34 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 34 v 77 (ObjNum 122 snap 20 seq_num 122) dirty exists 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:257: left oid 34 (ObjNum 122 snap 20 seq_num 122) 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:258: finishing write tid 1 to smithi17529310-43 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:258: finishing write tid 2 to smithi17529310-43 2024-09-15T01:33:13.275 INFO:tasks.rados.rados.0.smithi175.stdout:258: finishing write tid 3 to smithi17529310-43 2024-09-15T01:33:13.276 INFO:tasks.rados.rados.0.smithi175.stdout:258: finishing write tid 4 to smithi17529310-43 2024-09-15T01:33:13.276 INFO:tasks.rados.rados.0.smithi175.stdout:258: finishing write tid 5 to smithi17529310-43 2024-09-15T01:33:13.276 INFO:tasks.rados.rados.0.smithi175.stdout:258: finishing write tid 6 to smithi17529310-43 2024-09-15T01:33:13.276 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 43 v 48 (ObjNum 123 snap 20 seq_num 123) dirty exists 2024-09-15T01:33:13.276 INFO:tasks.rados.rados.0.smithi175.stdout:258: left oid 43 (ObjNum 123 snap 20 seq_num 123) 2024-09-15T01:33:13.276 INFO:tasks.rados.rados.0.smithi175.stdout:259: finishing write tid 1 to smithi17529310-4 2024-09-15T01:33:13.276 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:13.266+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:13.277 INFO:tasks.rados.rados.0.smithi175.stdout:257: done (3 left) 2024-09-15T01:33:13.277 INFO:tasks.rados.rados.0.smithi175.stdout:258: done (2 left) 2024-09-15T01:33:13.278 INFO:tasks.rados.rados.0.smithi175.stdout:260: done (1 left) 2024-09-15T01:33:13.278 INFO:tasks.rados.rados.0.smithi175.stdout:261: rollback oid 32 current snap is 20 2024-09-15T01:33:13.278 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 32 to 16 2024-09-15T01:33:13.278 INFO:tasks.rados.rados.0.smithi175.stdout:262: rmattr oid 44 current snap is 20 2024-09-15T01:33:13.278 INFO:tasks.rados.rados.0.smithi175.stdout:262: done (2 left) 2024-09-15T01:33:13.278 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 17 current snap is 20 2024-09-15T01:33:13.278 INFO:tasks.rados.rados.0.smithi175.stdout:263: seq_num 126 ranges {2113536=573440,2686976=622592,3309568=622592,3932160=311296} 2024-09-15T01:33:13.288 INFO:tasks.rados.rados.0.smithi175.stdout:263: writing smithi17529310-17 from 2113536 to 2686976 tid 1 2024-09-15T01:33:13.289 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:13.290 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:13.290 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:13.290 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:13.292 INFO:tasks.rados.rados.0.smithi175.stdout:263: writing smithi17529310-17 from 2686976 to 3309568 tid 2 2024-09-15T01:33:13.295 INFO:tasks.rados.rados.0.smithi175.stdout:263: writing smithi17529310-17 from 3309568 to 3932160 tid 3 2024-09-15T01:33:13.297 INFO:tasks.rados.rados.0.smithi175.stdout:263: writing smithi17529310-17 from 3932160 to 4243456 tid 4 2024-09-15T01:33:13.297 INFO:tasks.rados.rados.0.smithi175.stdout:264: read oid 26 snap 13 2024-09-15T01:33:13.297 INFO:tasks.rados.rados.0.smithi175.stdout:264: expect (ObjNum 103 snap 10 seq_num 103) 2024-09-15T01:33:13.298 INFO:tasks.rados.rados.0.smithi175.stdout:265: copy_from oid 29 from oid 43 current snap is 20 2024-09-15T01:33:13.298 INFO:tasks.rados.rados.0.smithi175.stdout:259: finishing write tid 2 to smithi17529310-4 2024-09-15T01:33:13.298 INFO:tasks.rados.rados.0.smithi175.stdout:266: setattr oid 37 current snap is 20 2024-09-15T01:33:13.298 INFO:tasks.rados.rados.0.smithi175.stdout:259: finishing write tid 3 to smithi17529310-4 2024-09-15T01:33:13.298 INFO:tasks.rados.rados.0.smithi175.stdout:259: finishing write tid 4 to smithi17529310-4 2024-09-15T01:33:13.298 INFO:tasks.rados.rados.0.smithi175.stdout:261: finishing rollback tid 0 to smithi17529310-32 2024-09-15T01:33:13.298 INFO:tasks.rados.rados.0.smithi175.stdout:261: finishing rollback tid 1 to smithi17529310-32 2024-09-15T01:33:13.298 INFO:tasks.rados.rados.0.smithi175.stdout:261: finishing rollback tid 2 to smithi17529310-32 2024-09-15T01:33:13.298 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 32 v 73 (ObjNum 86 snap 7 seq_num 86) dirty exists 2024-09-15T01:33:13.299 INFO:tasks.rados.rados.0.smithi175.stdout:261: done (5 left) 2024-09-15T01:33:13.299 INFO:tasks.rados.rados.0.smithi175.stdout:267: copy_from oid 16 from oid 31 current snap is 20 2024-09-15T01:33:13.299 INFO:tasks.rados.rados.0.smithi175.stdout:268: rollback oid 19 current snap is 20 2024-09-15T01:33:13.299 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 19 to 17 2024-09-15T01:33:13.300 INFO:tasks.rados.rados.0.smithi175.stdout:269: copy_from oid 46 from oid 49 current snap is 20 2024-09-15T01:33:13.300 INFO:tasks.rados.rados.0.smithi175.stdout:270: copy_from oid 6 from oid 42 current snap is 20 2024-09-15T01:33:13.300 INFO:tasks.rados.rados.0.smithi175.stdout:271: read oid 20 snap 16 2024-09-15T01:33:13.300 INFO:tasks.rados.rados.0.smithi175.stdout:271: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-15T01:33:13.300 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 50 current snap is 20 2024-09-15T01:33:13.300 INFO:tasks.rados.rados.0.smithi175.stdout:263: finishing write tid 1 to smithi17529310-17 2024-09-15T01:33:13.300 INFO:tasks.rados.rados.0.smithi175.stdout:272: seq_num 127 ranges {2457600=589824,3047424=753664,3801088=475136,4276224=327680} 2024-09-15T01:33:13.301 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:33:13.317 INFO:tasks.rados.rados.0.smithi175.stdout:272: writing smithi17529310-50 from 2457600 to 3047424 tid 1 2024-09-15T01:33:13.321 INFO:tasks.rados.rados.0.smithi175.stdout:272: writing smithi17529310-50 from 3047424 to 3801088 tid 2 2024-09-15T01:33:13.324 INFO:tasks.rados.rados.0.smithi175.stdout:272: writing smithi17529310-50 from 3801088 to 4276224 tid 3 2024-09-15T01:33:13.327 INFO:tasks.rados.rados.0.smithi175.stdout:272: writing smithi17529310-50 from 4276224 to 4603904 tid 4 2024-09-15T01:33:13.327 INFO:tasks.rados.rados.0.smithi175.stdout:273: setattr oid 2 current snap is 20 2024-09-15T01:33:13.327 INFO:tasks.rados.rados.0.smithi175.stdout:263: finishing write tid 2 to smithi17529310-17 2024-09-15T01:33:13.327 INFO:tasks.rados.rados.0.smithi175.stdout:263: finishing write tid 3 to smithi17529310-17 2024-09-15T01:33:13.327 INFO:tasks.rados.rados.0.smithi175.stdout:271: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-15T01:33:13.337 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 37 v 50 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-15T01:33:13.338 INFO:tasks.rados.rados.0.smithi175.stdout:263: finishing write tid 4 to smithi17529310-17 2024-09-15T01:33:13.338 INFO:tasks.rados.rados.0.smithi175.stdout:263: finishing write tid 5 to smithi17529310-17 2024-09-15T01:33:13.338 INFO:tasks.rados.rados.0.smithi175.stdout:264: expect (ObjNum 103 snap 10 seq_num 103) 2024-09-15T01:33:13.370 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:13.368+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:13.400 INFO:tasks.rados.rados.0.smithi175.stdout:268: finishing rollback tid 0 to smithi17529310-19 2024-09-15T01:33:13.401 INFO:tasks.rados.rados.0.smithi175.stdout:268: finishing rollback tid 1 to smithi17529310-19 2024-09-15T01:33:13.401 INFO:tasks.rados.rados.0.smithi175.stdout:269: finishing copy_from racing read to smithi17529310-46 2024-09-15T01:33:13.401 INFO:tasks.rados.rados.0.smithi175.stdout:268: finishing rollback tid 2 to smithi17529310-19 2024-09-15T01:33:13.401 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 19 v 59 (ObjNum 105 snap 10 seq_num 105) dirty exists 2024-09-15T01:33:13.401 INFO:tasks.rados.rados.0.smithi175.stdout:269: finishing copy_from to smithi17529310-46 2024-09-15T01:33:13.401 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 46 v 39 (ObjNum 101 snap 10 seq_num 101) dirty exists 2024-09-15T01:33:13.401 INFO:tasks.rados.rados.0.smithi175.stdout:267: finishing copy_from racing read to smithi17529310-16 2024-09-15T01:33:13.401 INFO:tasks.rados.rados.0.smithi175.stdout:265: finishing copy_from racing read to smithi17529310-29 2024-09-15T01:33:13.401 INFO:tasks.rados.rados.0.smithi175.stdout:270: finishing copy_from racing read to smithi17529310-6 2024-09-15T01:33:13.401 INFO:tasks.rados.rados.0.smithi175.stdout:259: finishing write tid 5 to smithi17529310-4 2024-09-15T01:33:13.401 INFO:tasks.rados.rados.0.smithi175.stdout:264: done (11 left) 2024-09-15T01:33:13.401 INFO:tasks.rados.rados.0.smithi175.stdout:266: done (10 left) 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:268: done (9 left) 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:269: done (8 left) 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:271: done (7 left) 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:274: delete oid 11 current snap is 20 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:272: finishing write tid 1 to smithi17529310-50 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:263: finishing write tid 6 to smithi17529310-17 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 17 v 134 (ObjNum 126 snap 20 seq_num 126) dirty exists 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:263: left oid 17 (ObjNum 126 snap 20 seq_num 126) 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:267: finishing copy_from to smithi17529310-16 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 16 v 53 (ObjNum 85 snap 7 seq_num 85) dirty exists 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:259: finishing write tid 6 to smithi17529310-4 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 4 v 64 (ObjNum 124 snap 20 seq_num 124) dirty exists 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:259: left oid 4 (ObjNum 124 snap 20 seq_num 124) 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:272: finishing write tid 2 to smithi17529310-50 2024-09-15T01:33:13.402 INFO:tasks.rados.rados.0.smithi175.stdout:270: finishing copy_from to smithi17529310-6 2024-09-15T01:33:13.403 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 6 v 59 (ObjNum 105 snap 10 seq_num 105) dirty exists 2024-09-15T01:33:13.403 INFO:tasks.rados.rados.0.smithi175.stdout:272: finishing write tid 3 to smithi17529310-50 2024-09-15T01:33:13.403 INFO:tasks.rados.rados.0.smithi175.stdout:272: finishing write tid 4 to smithi17529310-50 2024-09-15T01:33:13.403 INFO:tasks.rados.rados.0.smithi175.stdout:272: finishing write tid 5 to smithi17529310-50 2024-09-15T01:33:13.403 INFO:tasks.rados.rados.0.smithi175.stdout:272: finishing write tid 6 to smithi17529310-50 2024-09-15T01:33:13.403 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 50 v 29 (ObjNum 127 snap 20 seq_num 127) dirty exists 2024-09-15T01:33:13.403 INFO:tasks.rados.rados.0.smithi175.stdout:272: left oid 50 (ObjNum 127 snap 20 seq_num 127) 2024-09-15T01:33:13.403 INFO:tasks.rados.rados.0.smithi175.stdout:265: finishing copy_from to smithi17529310-29 2024-09-15T01:33:13.403 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 29 v 48 (ObjNum 123 snap 20 seq_num 123) dirty exists 2024-09-15T01:33:13.404 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:13.404 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:13.404 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:13.404 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:13.404 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:13.404 INFO:tasks.rados.rados.0.smithi175.stdout:259: done (7 left) 2024-09-15T01:33:13.404 INFO:tasks.rados.rados.0.smithi175.stdout:263: done (6 left) 2024-09-15T01:33:13.404 INFO:tasks.rados.rados.0.smithi175.stdout:265: done (5 left) 2024-09-15T01:33:13.404 INFO:tasks.rados.rados.0.smithi175.stdout:267: done (4 left) 2024-09-15T01:33:13.405 INFO:tasks.rados.rados.0.smithi175.stdout:270: done (3 left) 2024-09-15T01:33:13.405 INFO:tasks.rados.rados.0.smithi175.stdout:272: done (2 left) 2024-09-15T01:33:13.405 INFO:tasks.rados.rados.0.smithi175.stdout:274: done (1 left) 2024-09-15T01:33:13.405 INFO:tasks.rados.rados.0.smithi175.stdout:275: delete oid 35 current snap is 20 2024-09-15T01:33:13.412 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:33:13.447 INFO:tasks.rados.rados.0.smithi175.stdout:275: done (1 left) 2024-09-15T01:33:13.447 INFO:tasks.rados.rados.0.smithi175.stdout:276: copy_from oid 36 from oid 44 current snap is 20 2024-09-15T01:33:13.447 INFO:tasks.rados.rados.0.smithi175.stdout:277: snap_create 2024-09-15T01:33:13.456 INFO:tasks.rados.rados.0.smithi175.stdout:276: finishing copy_from racing read to smithi17529310-36 2024-09-15T01:33:13.457 INFO:tasks.rados.rados.0.smithi175.stdout:276: finishing copy_from to smithi17529310-36 2024-09-15T01:33:13.457 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 36 v 49 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-15T01:33:13.471 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:13.469+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 71, 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "d4803586-ed44-4154-8ee1-6b95ec3e17e7", 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-15T01:32:07.285177+0000", 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-15T01:33:13.072530+0000", 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-15T01:32:14.443323+0000", 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-15T01:33:08.068993+0000", 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5804032, 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout: "nodeep-scrub", 2024-09-15T01:33:13.501 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 5, 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 16, 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "jewel", 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "quincy", 2024-09-15T01:33:13.502 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-15T01:32:15.118634+0000", 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:33:13.503 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.504 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "24", 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-15T01:33:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-15T01:33:13.506 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-15T01:33:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-15T01:33:13.508 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-15T01:33:13.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.508 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-15T01:33:13.508 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-15T01:33:13.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.508 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-15T01:33:13.508 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-15T01:33:13.508 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-15T01:32:21.824131+0000", 2024-09-15T01:33:13.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-15T01:33:13.509 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "28", 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-15T01:33:13.510 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 3, 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 28, 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-15T01:33:13.511 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-15T01:33:13.512 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-15T01:33:13.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "unique_pool_0", 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-15T01:32:32.183257+0000", 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 3, 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 7, 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 5, 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 1, 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:33:13.514 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 16, 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 16, 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 16, 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 16, 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 16, 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "71", 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 35, 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 71, 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-15T01:33:13.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "shecprofile", 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-15T01:33:13.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 16384, 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-15T01:33:13.518 INFO:teuthology.orchestra.run.smithi042.stdout: "rados": {} 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "bec41583-a0a4-4712-a9ec-eec1e88fb60f", 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 69, 2024-09-15T01:33:13.519 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.521 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.521 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.521 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-15T01:33:13.521 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:13.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.521 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-15T01:33:13.521 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:13.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.522 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/28861", 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/28861", 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/28861", 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/28861", 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "2cca66be-8685-4a38-abf8-c864cd5434da", 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.524 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 0, 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6808", 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:13.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6809", 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6810", 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6811", 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6817", 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6819", 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6813", 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6815", 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.115:6809/28508", 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.115:6811/28508", 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.115:6819/28508", 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.115:6815/28508", 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.529 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b8756aac-3a3e-41ba-a6ef-497063cb44df", 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 66, 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.530 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6808", 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6809", 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6810", 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6811", 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6814", 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6816", 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6812", 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6813", 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.152:6809/28732", 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.152:6811/28732", 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.152:6816/28732", 2024-09-15T01:33:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.152:6813/28732", 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "554875f5-74c0-4b58-80ee-6ba93748f104", 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 69, 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6802", 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6803", 2024-09-15T01:33:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6804", 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6805", 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6808", 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6809", 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6806", 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6807", 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.175:6803/28835", 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.175:6805/28835", 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.175:6809/28835", 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.175:6807/28835", 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "3fe2c05a-4c22-4dd7-a8c6-282155c82eb8", 2024-09-15T01:33:13.540 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 69, 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-15T01:33:13.541 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.543 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.544 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/28860", 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/28860", 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/28860", 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/28860", 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "a301d1ce-3e31-4782-ade3-f9c0f1e6a95d", 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 31, 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6800", 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6801", 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.548 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6802", 2024-09-15T01:33:13.548 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:13.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.548 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6803", 2024-09-15T01:33:13.548 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:13.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.548 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.548 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6806", 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6807", 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6804", 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6805", 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.115:6801/28509", 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.115:6803/28509", 2024-09-15T01:33:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.115:6807/28509", 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.115:6805/28509", 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "c7d180be-21c5-4674-a9f2-10a1c7794871", 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.551 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 25, 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6800", 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.552 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6801", 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6802", 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.553 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6803", 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6806", 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6807", 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:13.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6804", 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6805", 2024-09-15T01:33:13.555 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.152:6801/28731", 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.152:6803/28731", 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.152:6807/28731", 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.152:6805/28731", 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-15T01:33:13.556 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "4d2429ea-772f-4fb9-a421-240f1399f16c", 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 31, 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6810", 2024-09-15T01:33:13.557 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6811", 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6812", 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6813", 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6816", 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6817", 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6814", 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.560 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6815", 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.175:6811/28836", 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.175:6813/28836", 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.175:6817/28836", 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.175:6815/28836", 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 8, 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "de7389ab-a29e-4177-9e94-484fc6a94102", 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 31, 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.564 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/28862", 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/28862", 2024-09-15T01:33:13.566 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/28862", 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/28862", 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 9, 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "4b4b44da-97d6-41f2-8fc6-57680b10aeb8", 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.567 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 0, 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6812", 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6814", 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6816", 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.569 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6818", 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6822", 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.570 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6823", 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6820", 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6821", 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.115:6814/28510", 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.115:6818/28510", 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.115:6823/28510", 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.115:6821/28510", 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 10, 2024-09-15T01:33:13.573 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "5f285a07-9057-4575-b911-b3977acd397c", 2024-09-15T01:33:13.573 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.573 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.573 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.573 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.573 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.573 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.573 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.573 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 0, 2024-09-15T01:33:13.573 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6815", 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6817", 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.575 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.575 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.575 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6818", 2024-09-15T01:33:13.575 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:13.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.575 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6819", 2024-09-15T01:33:13.575 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:13.575 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6822", 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6824", 2024-09-15T01:33:13.576 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6820", 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.577 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6821", 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.152:6817/28733", 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.152:6819/28733", 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.152:6824/28733", 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.152:6821/28733", 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 11, 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "dacc0f3f-a598-4055-92f4-d41102ceca83", 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 31, 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6826", 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6827", 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6828", 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6829", 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.581 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6832", 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6833", 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6830", 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6831", 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.175:6827/28837", 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.175:6829/28837", 2024-09-15T01:33:13.583 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.175:6833/28837", 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.175:6831/28837", 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 12, 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d218197b-d0db-410c-abff-8b849a21a555", 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.584 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 0, 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.585 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:13.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:13.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/28866", 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/28866", 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/28866", 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/28866", 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 13, 2024-09-15T01:33:13.589 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "c9bf5faf-f2ee-482e-b903-28c7718e2063", 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 54, 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.590 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6824", 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6825", 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6826", 2024-09-15T01:33:13.591 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6827", 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6830", 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:13.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6831", 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6828", 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:13.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6829", 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.115:6825/28511", 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.115:6827/28511", 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.115:6831/28511", 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.115:6829/28511", 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.594 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 14, 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "cc3f19cf-5a97-440b-a953-7727e6ed6a0d", 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 31, 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.595 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6823", 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6825", 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6826", 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6827", 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6830", 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6831", 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6828", 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6829", 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.152:6825/28734", 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.152:6827/28734", 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.152:6831/28734", 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.152:6829/28734", 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 15, 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "51290c23-e86d-4e05-9ac8-ca53d6a1eb80", 2024-09-15T01:33:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 31, 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6818", 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6819", 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6820", 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6821", 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6824", 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6825", 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6822", 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6823", 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.175:6819/28838", 2024-09-15T01:33:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.175:6821/28838", 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.175:6825/28838", 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.175:6823/28838", 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.539908+0000", 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.544884+0000", 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.607 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.614659+0000", 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.608 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.605046+0000", 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.503545+0000", 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.584788+0000", 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.563649+0000", 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.622071+0000", 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 8, 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.572859+0000", 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 9, 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.526526+0000", 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 10, 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.534845+0000", 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 11, 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.556801+0000", 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 12, 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.565526+0000", 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 13, 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.582172+0000", 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 14, 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.619730+0000", 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 15, 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.619890+0000", 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-15T01:33:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "shecprofile": { 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "c": "2", 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-device-class": "", 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-root": "default", 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "4", 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "3", 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "shecprofile", 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "shec", 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "multiple" 2024-09-15T01:33:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [ 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 21, 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 27, 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 33, 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [ 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 33, 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-15T01:33:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:13.621 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:13.622 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:33:13.626 INFO:tasks.thrashosds.thrasher:j is {'epoch': 71, 'fsid': 'd4803586-ed44-4154-8ee1-6b95ec3e17e7', 'created': '2024-09-15T01:32:07.285177+0000', 'modified': '2024-09-15T01:33:13.072530+0000', 'last_up_change': '2024-09-15T01:32:14.443323+0000', 'last_in_change': '2024-09-15T01:33:08.068993+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T01:32:15.118634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T01:32:21.824131+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 28, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-15T01:32:32.183257+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'bec41583-a0a4-4712-a9ec-eec1e88fb60f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 28861}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 28861}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 28861}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 28861}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 28861}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 28861}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 28861}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 28861}]}, 'public_addr': '172.21.15.42:6809/28861', 'cluster_addr': '172.21.15.42:6811/28861', 'heartbeat_back_addr': '172.21.15.42:6815/28861', 'heartbeat_front_addr': '172.21.15.42:6813/28861', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '2cca66be-8685-4a38-abf8-c864cd5434da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 28508}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 28508}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 28508}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 28508}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6817', 'nonce': 28508}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 28508}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6813', 'nonce': 28508}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 28508}]}, 'public_addr': '172.21.15.115:6809/28508', 'cluster_addr': '172.21.15.115:6811/28508', 'heartbeat_back_addr': '172.21.15.115:6819/28508', 'heartbeat_front_addr': '172.21.15.115:6815/28508', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'b8756aac-3a3e-41ba-a6ef-497063cb44df', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6808', 'nonce': 28732}, {'type': 'v1', 'addr': '172.21.15.152:6809', 'nonce': 28732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6810', 'nonce': 28732}, {'type': 'v1', 'addr': '172.21.15.152:6811', 'nonce': 28732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6814', 'nonce': 28732}, {'type': 'v1', 'addr': '172.21.15.152:6816', 'nonce': 28732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6812', 'nonce': 28732}, {'type': 'v1', 'addr': '172.21.15.152:6813', 'nonce': 28732}]}, 'public_addr': '172.21.15.152:6809/28732', 'cluster_addr': '172.21.15.152:6811/28732', 'heartbeat_back_addr': '172.21.15.152:6816/28732', 'heartbeat_front_addr': '172.21.15.152:6813/28732', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '554875f5-74c0-4b58-80ee-6ba93748f104', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6802', 'nonce': 28835}, {'type': 'v1', 'addr': '172.21.15.175:6803', 'nonce': 28835}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6804', 'nonce': 28835}, {'type': 'v1', 'addr': '172.21.15.175:6805', 'nonce': 28835}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6808', 'nonce': 28835}, {'type': 'v1', 'addr': '172.21.15.175:6809', 'nonce': 28835}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6806', 'nonce': 28835}, {'type': 'v1', 'addr': '172.21.15.175:6807', 'nonce': 28835}]}, 'public_addr': '172.21.15.175:6803/28835', 'cluster_addr': '172.21.15.175:6805/28835', 'heartbeat_back_addr': '172.21.15.175:6809/28835', 'heartbeat_front_addr': '172.21.15.175:6807/28835', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3fe2c05a-4c22-4dd7-a8c6-282155c82eb8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 28860}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 28860}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 28860}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 28860}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 28860}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 28860}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 28860}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 28860}]}, 'public_addr': '172.21.15.42:6801/28860', 'cluster_addr': '172.21.15.42:6803/28860', 'heartbeat_back_addr': '172.21.15.42:6807/28860', 'heartbeat_front_addr': '172.21.15.42:6805/28860', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a301d1ce-3e31-4782-ade3-f9c0f1e6a95d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 28509}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 28509}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 28509}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 28509}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 28509}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 28509}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 28509}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 28509}]}, 'public_addr': '172.21.15.115:6801/28509', 'cluster_addr': '172.21.15.115:6803/28509', 'heartbeat_back_addr': '172.21.15.115:6807/28509', 'heartbeat_front_addr': '172.21.15.115:6805/28509', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c7d180be-21c5-4674-a9f2-10a1c7794871', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6800', 'nonce': 28731}, {'type': 'v1', 'addr': '172.21.15.152:6801', 'nonce': 28731}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6802', 'nonce': 28731}, {'type': 'v1', 'addr': '172.21.15.152:6803', 'nonce': 28731}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6806', 'nonce': 28731}, {'type': 'v1', 'addr': '172.21.15.152:6807', 'nonce': 28731}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6804', 'nonce': 28731}, {'type': 'v1', 'addr': '172.21.15.152:6805', 'nonce': 28731}]}, 'public_addr': '172.21.15.152:6801/28731', 'cluster_addr': '172.21.15.152:6803/28731', 'heartbeat_back_addr': '172.21.15.152:6807/28731', 'heartbeat_front_addr': '172.21.15.152:6805/28731', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4d2429ea-772f-4fb9-a421-240f1399f16c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6810', 'nonce': 28836}, {'type': 'v1', 'addr': '172.21.15.175:6811', 'nonce': 28836}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6812', 'nonce': 28836}, {'type': 'v1', 'addr': '172.21.15.175:6813', 'nonce': 28836}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6816', 'nonce': 28836}, {'type': 'v1', 'addr': '172.21.15.175:6817', 'nonce': 28836}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6814', 'nonce': 28836}, {'type': 'v1', 'addr': '172.21.15.175:6815', 'nonce': 28836}]}, 'public_addr': '172.21.15.175:6811/28836', 'cluster_addr': '172.21.15.175:6813/28836', 'heartbeat_back_addr': '172.21.15.175:6817/28836', 'heartbeat_front_addr': '172.21.15.175:6815/28836', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': 'de7389ab-a29e-4177-9e94-484fc6a94102', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 28862}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 28862}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 28862}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 28862}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 28862}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 28862}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 28862}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 28862}]}, 'public_addr': '172.21.15.42:6817/28862', 'cluster_addr': '172.21.15.42:6819/28862', 'heartbeat_back_addr': '172.21.15.42:6823/28862', 'heartbeat_front_addr': '172.21.15.42:6821/28862', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '4b4b44da-97d6-41f2-8fc6-57680b10aeb8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 28510}, {'type': 'v1', 'addr': '172.21.15.115:6814', 'nonce': 28510}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 28510}, {'type': 'v1', 'addr': '172.21.15.115:6818', 'nonce': 28510}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 28510}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 28510}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 28510}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 28510}]}, 'public_addr': '172.21.15.115:6814/28510', 'cluster_addr': '172.21.15.115:6818/28510', 'heartbeat_back_addr': '172.21.15.115:6823/28510', 'heartbeat_front_addr': '172.21.15.115:6821/28510', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '5f285a07-9057-4575-b911-b3977acd397c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6815', 'nonce': 28733}, {'type': 'v1', 'addr': '172.21.15.152:6817', 'nonce': 28733}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6818', 'nonce': 28733}, {'type': 'v1', 'addr': '172.21.15.152:6819', 'nonce': 28733}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6822', 'nonce': 28733}, {'type': 'v1', 'addr': '172.21.15.152:6824', 'nonce': 28733}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6820', 'nonce': 28733}, {'type': 'v1', 'addr': '172.21.15.152:6821', 'nonce': 28733}]}, 'public_addr': '172.21.15.152:6817/28733', 'cluster_addr': '172.21.15.152:6819/28733', 'heartbeat_back_addr': '172.21.15.152:6824/28733', 'heartbeat_front_addr': '172.21.15.152:6821/28733', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'dacc0f3f-a598-4055-92f4-d41102ceca83', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6826', 'nonce': 28837}, {'type': 'v1', 'addr': '172.21.15.175:6827', 'nonce': 28837}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6828', 'nonce': 28837}, {'type': 'v1', 'addr': '172.21.15.175:6829', 'nonce': 28837}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 28837}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 28837}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6830', 'nonce': 28837}, {'type': 'v1', 'addr': '172.21.15.175:6831', 'nonce': 28837}]}, 'public_addr': '172.21.15.175:6827/28837', 'cluster_addr': '172.21.15.175:6829/28837', 'heartbeat_back_addr': '172.21.15.175:6833/28837', 'heartbeat_front_addr': '172.21.15.175:6831/28837', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': 'd218197b-d0db-410c-abff-8b849a21a555', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 28866}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 28866}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 28866}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 28866}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 28866}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 28866}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 28866}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 28866}]}, 'public_addr': '172.21.15.42:6825/28866', 'cluster_addr': '172.21.15.42:6827/28866', 'heartbeat_back_addr': '172.21.15.42:6831/28866', 'heartbeat_front_addr': '172.21.15.42:6829/28866', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': 'c9bf5faf-f2ee-482e-b903-28c7718e2063', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 28511}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 28511}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 28511}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 28511}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 28511}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 28511}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 28511}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 28511}]}, 'public_addr': '172.21.15.115:6825/28511', 'cluster_addr': '172.21.15.115:6827/28511', 'heartbeat_back_addr': '172.21.15.115:6831/28511', 'heartbeat_front_addr': '172.21.15.115:6829/28511', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': 'cc3f19cf-5a97-440b-a953-7727e6ed6a0d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6823', 'nonce': 28734}, {'type': 'v1', 'addr': '172.21.15.152:6825', 'nonce': 28734}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6826', 'nonce': 28734}, {'type': 'v1', 'addr': '172.21.15.152:6827', 'nonce': 28734}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6830', 'nonce': 28734}, {'type': 'v1', 'addr': '172.21.15.152:6831', 'nonce': 28734}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6828', 'nonce': 28734}, {'type': 'v1', 'addr': '172.21.15.152:6829', 'nonce': 28734}]}, 'public_addr': '172.21.15.152:6825/28734', 'cluster_addr': '172.21.15.152:6827/28734', 'heartbeat_back_addr': '172.21.15.152:6831/28734', 'heartbeat_front_addr': '172.21.15.152:6829/28734', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '51290c23-e86d-4e05-9ac8-ca53d6a1eb80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6818', 'nonce': 28838}, {'type': 'v1', 'addr': '172.21.15.175:6819', 'nonce': 28838}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6820', 'nonce': 28838}, {'type': 'v1', 'addr': '172.21.15.175:6821', 'nonce': 28838}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6824', 'nonce': 28838}, {'type': 'v1', 'addr': '172.21.15.175:6825', 'nonce': 28838}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6822', 'nonce': 28838}, {'type': 'v1', 'addr': '172.21.15.175:6823', 'nonce': 28838}]}, 'public_addr': '172.21.15.175:6819/28838', 'cluster_addr': '172.21.15.175:6821/28838', 'heartbeat_back_addr': '172.21.15.175:6825/28838', 'heartbeat_front_addr': '172.21.15.175:6823/28838', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.539908+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.544884+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.614659+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.605046+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.503545+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.584788+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.563649+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.622071+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.572859+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.526526+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.534845+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.556801+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.565526+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.582172+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.619730+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.619890+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 21, 'length': 1}, {'begin': 27, 'length': 1}, {'begin': 33, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 33, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T01:33:13.626 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:33:13.626 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:13.621+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:13.627 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:13.627 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:13.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:13.627 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:13.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.627 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 44/31 ECSubWrite(tid=82, reqid=client.4477.0:281, at_version=44'36, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:13.627 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.243189+0000", 2024-09-15T01:33:13.627 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.266180465000001, 2024-09-15T01:33:13.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0097687209999999993, 2024-09-15T01:33:13.627 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243189+0000", 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243189+0000", 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243190+0000", 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1800000000000004e-07 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243219+0000", 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8903000000000001e-05 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243259+0000", 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0348000000000001e-05 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243267+0000", 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2490000000000001e-06 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243347+0000", 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9412000000000005e-05 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243358+0000", 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1447999999999999e-05 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243828+0000", 2024-09-15T01:33:13.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00046974499999999998 2024-09-15T01:33:13.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243876+0000", 2024-09-15T01:33:13.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7651999999999997e-05 2024-09-15T01:33:13.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245401+0000", 2024-09-15T01:33:13.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015253230000000001 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245443+0000", 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2135999999999998e-05 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252958+0000", 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0075145869999999997 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.262476585999998, 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011534543, 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246894+0000", 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0500000000000002e-07 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:13.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2585000000000001e-05 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246951+0000", 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4552999999999999e-05 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246961+0000", 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0516000000000001e-05 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247025+0000", 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3540999999999994e-05 2024-09-15T01:33:13.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247039+0000", 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4484000000000001e-05 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247338+0000", 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00029853699999999998 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248324+0000", 2024-09-15T01:33:13.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0009866059999999999 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248375+0000", 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0337000000000003e-05 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248387+0000", 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2081e-05 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258427+0000", 2024-09-15T01:33:13.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010040498 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as6 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.261832447, 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010349413, 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247538+0000", 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0130000000000001e-06 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247552+0000", 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3723999999999999e-05 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247588+0000", 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5989999999999999e-05 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247595+0000", 2024-09-15T01:33:13.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5789999999999998e-06 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247648+0000", 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2757000000000003e-05 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247654+0000", 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3629999999999999e-06 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247737+0000", 2024-09-15T01:33:13.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2886000000000006e-05 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248326+0000", 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058858900000000004 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248380+0000", 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3832999999999998e-05 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248401+0000", 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0843e-05 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257886+0000", 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0094858359999999992 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.250301093000001, 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015398044, 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.643 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259069+0000", 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2399999999999996e-07 2024-09-15T01:33:13.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259095+0000", 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5527000000000001e-05 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259133+0000", 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8211999999999999e-05 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259140+0000", 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9330000000000004e-06 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259189+0000", 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8797000000000001e-05 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259200+0000", 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0932e-05 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259425+0000", 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000225505 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259453+0000", 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8152000000000001e-05 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.261606+0000", 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021525870000000001 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.261621+0000", 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4829e-05 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274466+0000", 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012845646 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 44/31 ECSubWrite(tid=39, reqid=client.4477.0:283, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:13.648 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.240976883999998, 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014946894, 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:13.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0299999999999998e-07 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268593+0000", 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000199278 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268633+0000", 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0367000000000002e-05 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268640+0000", 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5100000000000001e-06 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268742+0000", 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010184899999999999 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268752+0000", 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9299999999999998e-06 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269007+0000", 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000255016 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269035+0000", 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8039999999999999e-05 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271592+0000", 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025566260000000002 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271610+0000", 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7629999999999999e-05 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283339+0000", 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011729844999999999 2024-09-15T01:33:13.653 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.6s6 44/38 ECSubWrite(tid=22, reqid=client.4477.0:293, at_version=44'14, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.238193363000001, 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012212746, 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271177+0000", 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271394+0000", 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021719400000000001 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271421+0000", 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6842000000000002e-05 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271426+0000", 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9010000000000002e-06 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271464+0000", 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7928e-05 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271476+0000", 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1979e-05 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271800+0000", 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032464100000000002 2024-09-15T01:33:13.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271832+0000", 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1501999999999998e-05 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271921+0000", 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9514999999999999e-05 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271940+0000", 2024-09-15T01:33:13.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8436000000000001e-05 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283389+0000", 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011449086000000001 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 55/31 ECSubWrite(tid=160, reqid=client.4477.0:512, at_version=55'53, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.433692689000001, 2024-09-15T01:33:13.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011287287999999999, 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075698+0000", 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0384999999999999e-05 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075734+0000", 2024-09-15T01:33:13.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6300999999999997e-05 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075743+0000", 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9549999999999998e-06 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075803+0000", 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0304999999999999e-05 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075813+0000", 2024-09-15T01:33:13.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9539999999999999e-06 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075916+0000", 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102534 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075925+0000", 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1509999999999993e-06 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075971+0000", 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6276999999999998e-05 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075995+0000", 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3176999999999998e-05 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.086964+0000", 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010969504999999999 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.664 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 55/31 ECSubWrite(tid=161, reqid=client.4477.0:513, at_version=55'54, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.433056245, 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010838802, 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076314+0000", 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076329+0000", 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5505000000000001e-05 2024-09-15T01:33:13.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076356+0000", 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6324000000000001e-05 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076362+0000", 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2820000000000002e-06 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076409+0000", 2024-09-15T01:33:13.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7190000000000001e-05 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076419+0000", 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9450000000000005e-06 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076491+0000", 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1680999999999994e-05 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076497+0000", 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3810000000000004e-06 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076958+0000", 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00046042499999999998 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076971+0000", 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2999999999999999e-05 2024-09-15T01:33:13.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.087152+0000", 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010181366000000001 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 55/31 ECSubWrite(tid=56, reqid=client.4477.0:518, at_version=55'46, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.076867+0000", 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.432502450000001, 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010241029, 2024-09-15T01:33:13.670 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076867+0000", 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076867+0000", 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076868+0000", 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9500000000000001e-07 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076882+0000", 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4357e-05 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076915+0000", 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2954999999999999e-05 2024-09-15T01:33:13.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076920+0000", 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1270000000000002e-06 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076957+0000", 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6665000000000002e-05 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076966+0000", 2024-09-15T01:33:13.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3139999999999993e-06 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.077038+0000", 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1323999999999997e-05 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.077045+0000", 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4839999999999998e-06 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.077372+0000", 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032736700000000001 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.077388+0000", 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5744999999999999e-05 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.087108+0000", 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0097197959999999993 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.675 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 59/31 ECSubWrite(tid=179, reqid=client.4477.0:552, at_version=59'58, trim_to=0'0, roll_forward_to=56'56))", 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 12.417886976, 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010784815, 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091483+0000", 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2999999999999999e-07 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091650+0000", 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016715 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091673+0000", 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3434999999999998e-05 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091680+0000", 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7329999999999997e-06 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091721+0000", 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1088999999999999e-05 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091729+0000", 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9170000000000006e-06 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091940+0000", 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021130000000000001 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.092580+0000", 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00063981200000000002 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093157+0000", 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057673800000000003 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093178+0000", 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0911e-05 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.102267+0000", 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090895000000000004 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 66/31 ECSubWrite(tid=200, reqid=client.4477.0:607, at_version=66'106, trim_to=0'0, roll_forward_to=63'103))", 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:08.082552+0000", 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.4268170500000004, 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0096137429999999992, 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.681 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082552+0000", 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082552+0000", 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082553+0000", 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8299999999999996e-07 2024-09-15T01:33:13.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082573+0000", 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0193000000000001e-05 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082593+0000", 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9364e-05 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082600+0000", 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8390000000000001e-06 2024-09-15T01:33:13.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082645+0000", 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5524e-05 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082655+0000", 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8800000000000003e-06 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082759+0000", 2024-09-15T01:33:13.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010453799999999999 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082843+0000", 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3442000000000005e-05 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082918+0000", 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4723999999999998e-05 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.083172+0000", 2024-09-15T01:33:13.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025431999999999999 2024-09-15T01:33:13.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.092166+0000", 2024-09-15T01:33:13.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0089941359999999998 2024-09-15T01:33:13.686 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.686 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.686 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.688 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 66/31 ECSubWrite(tid=201, reqid=client.4477.0:608, at_version=66'107, trim_to=0'0, roll_forward_to=63'103))", 2024-09-15T01:33:13.688 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:08.082922+0000", 2024-09-15T01:33:13.688 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.4264472619999999, 2024-09-15T01:33:13.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.009397252, 2024-09-15T01:33:13.688 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.688 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.688 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082922+0000", 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082922+0000", 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082923+0000", 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5499999999999997e-07 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082942+0000", 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8936e-05 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082962+0000", 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9766999999999999e-05 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.082969+0000", 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7349999999999996e-06 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.083014+0000", 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4876000000000002e-05 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.083023+0000", 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0189999999999995e-06 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.083137+0000", 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011368 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.083154+0000", 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6838e-05 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.083214+0000", 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0248999999999999e-05 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.083226+0000", 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1722999999999999e-05 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.092319+0000", 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090935740000000001 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 66/31 ECSubWrite(tid=64, reqid=client.4477.0:609, at_version=66'56, trim_to=0'0, roll_forward_to=66'52))", 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:08.087598+0000", 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.4217714690000003, 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010297796999999999, 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.087598+0000", 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.087598+0000", 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.087599+0000", 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1100000000000005e-07 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088049+0000", 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045032600000000002 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088087+0000", 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8138e-05 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088095+0000", 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.34e-06 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088153+0000", 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-05 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088165+0000", 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1386000000000001e-05 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088485+0000", 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032002700000000002 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088493+0000", 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8550000000000008e-06 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088596+0000", 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102278 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088613+0000", 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7757e-05 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.097896+0000", 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0092822389999999994 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 71/31 ECSubWrite(tid=229, reqid=client.4477.0:651, at_version=71'127, trim_to=0'0, roll_forward_to=71'124))", 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.208012+0000", 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.30135706499999998, 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.016322817, 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.698 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208012+0000", 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208012+0000", 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208013+0000", 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3099999999999997e-07 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208033+0000", 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9539e-05 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208064+0000", 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1705999999999998e-05 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208073+0000", 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3750000000000003e-06 2024-09-15T01:33:13.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208136+0000", 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3736000000000001e-05 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208146+0000", 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9029999999999999e-06 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208264+0000", 2024-09-15T01:33:13.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117372 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208272+0000", 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.755e-06 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208300+0000", 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7875999999999999e-05 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208320+0000", 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9791000000000001e-05 2024-09-15T01:33:13.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224335+0000", 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.016015033000000001 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 71/31 ECSubWrite(tid=231, reqid=client.4477.0:652, at_version=71'128, trim_to=0'0, roll_forward_to=71'124))", 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.209071+0000", 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.30029828800000002, 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015677593, 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.703 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209071+0000", 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209071+0000", 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209072+0000", 2024-09-15T01:33:13.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1399999999999997e-07 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209084+0000", 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1917e-05 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209402+0000", 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00031803499999999999 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209407+0000", 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3179999999999997e-06 2024-09-15T01:33:13.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209454+0000", 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6981000000000003e-05 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209464+0000", 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0037999999999999e-05 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209592+0000", 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012783 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209599+0000", 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9489999999999999e-06 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209779+0000", 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017987 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209797+0000", 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8071e-05 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224749+0000", 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.01495207 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 71/31 ECSubWrite(tid=232, reqid=client.4477.0:652, at_version=0'0, trim_to=0'0, roll_forward_to=71'128))", 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.29888869299999998, 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014733626, 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8999999999999996e-07 2024-09-15T01:33:13.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210491+0000", 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7669999999999998e-06 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210530+0000", 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9039999999999999e-05 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210538+0000", 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2029999999999992e-06 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210601+0000", 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3069000000000002e-05 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210612+0000", 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0372e-05 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210761+0000", 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000149044 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210833+0000", 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2084999999999998e-05 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210854+0000", 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1424999999999999e-05 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210863+0000", 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1430000000000004e-06 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.225214+0000", 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014350887999999999 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=233, reqid=client.4477.0:653, at_version=0'0, trim_to=0'0, roll_forward_to=71'67))", 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.2988749, 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013895739000000001, 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0599999999999999e-07 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210501+0000", 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2310000000000001e-06 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210562+0000", 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0612000000000002e-05 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210570+0000", 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4109999999999996e-06 2024-09-15T01:33:13.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210619+0000", 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9419000000000002e-05 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210625+0000", 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5339999999999999e-06 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210691+0000", 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5905e-05 2024-09-15T01:33:13.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210698+0000", 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0910000000000004e-06 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210831+0000", 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000133065 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210844+0000", 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3349e-05 2024-09-15T01:33:13.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224390+0000", 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013545916 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=234, reqid=client.4477.0:654, at_version=71'68, trim_to=0'0, roll_forward_to=71'67))", 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.29772447600000002, 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013353726, 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.718 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:13.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2900000000000004e-07 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212326+0000", 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00068024600000000002 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212354+0000", 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8555999999999999e-05 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212360+0000", 2024-09-15T01:33:13.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.755e-06 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212397+0000", 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7086999999999999e-05 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212404+0000", 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7190000000000004e-06 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212510+0000", 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010600800000000001 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212515+0000", 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3159999999999996e-06 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213020+0000", 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000504854 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213032+0000", 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2048999999999999e-05 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224999+0000", 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011966607000000001 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=235, reqid=client.4477.0:654, at_version=0'0, trim_to=0'0, roll_forward_to=71'68))", 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:13.723 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.29581764399999999, 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011535082, 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4799999999999998e-07 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213560+0000", 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213582+0000", 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2524e-05 2024-09-15T01:33:13.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213588+0000", 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9009999999999999e-06 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213624+0000", 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5725000000000001e-05 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213629+0000", 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0679999999999996e-06 2024-09-15T01:33:13.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213684+0000", 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4722999999999998e-05 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213690+0000", 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4489999999999996e-06 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.215563+0000", 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018726680000000001 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.215576+0000", 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3021e-05 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.225087+0000", 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0095110449999999992 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=236, reqid=client.4477.0:655, at_version=71'69, trim_to=0'0, roll_forward_to=71'68))", 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.215629+0000", 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.29374081800000001, 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0096407920000000005, 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.215629+0000", 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.215629+0000", 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.215629+0000", 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0500000000000003e-07 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.216165+0000", 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053553900000000002 2024-09-15T01:33:13.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.216201+0000", 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5886000000000002e-05 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.216207+0000", 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7939999999999997e-06 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.216269+0000", 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1241999999999994e-05 2024-09-15T01:33:13.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.216290+0000", 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1639000000000001e-05 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.216439+0000", 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014835 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.216457+0000", 2024-09-15T01:33:13.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8847000000000001e-05 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.216562+0000", 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000104601 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.216830+0000", 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00026794900000000002 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:13.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.225269+0000", 2024-09-15T01:33:13.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00843934 2024-09-15T01:33:13.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.734 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:13.734 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:13.734 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:13.734 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-15T01:33:13.736 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:13.733+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:13.791 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:13.792 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:13.811 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:13.811 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:13.830 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:13.830 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:13.834 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:13.834 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:13.834 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:13.834 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:13.837 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:13.834+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:13.843 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-15T01:33:13.851 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:13.851 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:13.869 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:13.869 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:13.888 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:13.888 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:13.907 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:13.907 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:13.926 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:13.926 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:13.937 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:13.936+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:13.939 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:13.940 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:13.940 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:13.940 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:13.940 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:13.944 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:13.944 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:13.948 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-15T01:33:13.963 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:13.963 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:13.982 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:13.982 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:14.001 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:14.001 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:33:14.018 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:33:14.020 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:14.020 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:14.029 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-09-15T01:33:13.080227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":75563571,"num_objects":59,"num_object_clones":14,"num_object_copies":393,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":1,"num_read":359,"num_read_kb":50153,"num_write":659,"num_write_kb":107618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":20021248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":832,"ondisk_log_size":832,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":12958076,"kb_used_data":1485921972,"kb_used_omap":19816,"kb_used_meta":0,"kb_avail":1485915780,"statfs":{"total":1534846828544,"available":1521577758720,"internally_reserved":6340608,"allocated":1521584099328,"data_stored":1521584099328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20291649,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":29,"commit_latency_ns":0,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1687552,"num_objects":-8,"num_object_clones":-6,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8,"num_whiteouts":-2,"num_read":3,"num_read_kb":2,"num_write":13,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000782"},"pg_stats":[{"pgid":"3.f","version":"66'113","reported_seq":218,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T01:33:08.960023+0000","last_change":"2024-09-15T01:33:08.960023+0000","last_active":"2024-09-15T01:33:08.960023+0000","last_peered":"2024-09-15T01:33:08.960023+0000","last_clean":"2024-09-15T01:33:08.960023+0000","last_became_active":"2024-09-15T01:32:34.487273+0000","last_became_peered":"2024-09-15T01:32:34.487273+0000","last_unstale":"2024-09-15T01:33:08.960023+0000","last_undegraded":"2024-09-15T01:33:08.960023+0000","last_fullsized":"2024-09-15T01:33:08.960023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:42.230488+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.88014949099999995,"stat_sum":{"num_bytes":15237120,"num_objects":9,"num_object_clones":3,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":293,"num_write":91,"num_write_kb":20112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"15","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":146,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.328564+0000","last_change":"2024-09-15T01:33:09.328564+0000","last_active":"2024-09-15T01:33:09.328564+0000","last_peered":"2024-09-15T01:33:09.328564+0000","last_clean":"2024-09-15T01:33:09.328564+0000","last_became_active":"2024-09-15T01:33:09.327259+0000","last_became_peered":"2024-09-15T01:33:09.327259+0000","last_unstale":"2024-09-15T01:33:09.328564+0000","last_undegraded":"2024-09-15T01:33:09.328564+0000","last_fullsized":"2024-09-15T01:33:09.328564+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:57.127309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077024600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,1,11,8,6,12],"acting":[2,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"15","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"3.d","version":"66'59","reported_seq":165,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.078494+0000","last_change":"2024-09-15T01:33:08.094877+0000","last_active":"2024-09-15T01:33:09.078494+0000","last_peered":"2024-09-15T01:33:09.078494+0000","last_clean":"2024-09-15T01:33:09.078494+0000","last_became_active":"2024-09-15T01:32:34.487040+0000","last_became_peered":"2024-09-15T01:32:34.487040+0000","last_unstale":"2024-09-15T01:33:09.078494+0000","last_undegraded":"2024-09-15T01:33:09.078494+0000","last_fullsized":"2024-09-15T01:33:09.078494+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.966217+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015912724999999999,"stat_sum":{"num_bytes":4407296,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":18,"num_read_kb":872,"num_write":45,"num_write_kb":5360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"15","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"3.c","version":"66'62","reported_seq":169,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T01:33:08.087206+0000","last_change":"2024-09-15T01:33:08.087206+0000","last_active":"2024-09-15T01:33:08.087206+0000","last_peered":"2024-09-15T01:33:08.087206+0000","last_clean":"2024-09-15T01:33:08.087206+0000","last_became_active":"2024-09-15T01:32:34.487072+0000","last_became_peered":"2024-09-15T01:32:34.487072+0000","last_unstale":"2024-09-15T01:33:08.087206+0000","last_undegraded":"2024-09-15T01:33:08.087206+0000","last_fullsized":"2024-09-15T01:33:08.087206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.186197+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0093158789999999991,"stat_sum":{"num_bytes":7864320,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":53,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"15","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"3.b","version":"53'23","reported_seq":169,"reported_epoch":68,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-15T01:33:10.087143+0000","last_change":"2024-09-15T01:33:10.084730+0000","last_active":"2024-09-15T01:33:08.076477+0000","last_peered":"2024-09-15T01:33:07.070445+0000","last_clean":"2024-09-15T01:33:07.070445+0000","last_became_active":"2024-09-15T01:32:41.206211+0000","last_became_peered":"2024-09-15T01:32:41.206211+0000","last_unstale":"2024-09-15T01:33:10.087143+0000","last_undegraded":"2024-09-15T01:33:10.082035+0000","last_fullsized":"2024-09-15T01:33:10.081512+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:37.545285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002900386,"stat_sum":{"num_bytes":2457600,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":10,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,2147483647],"avail_no_missing":["4(0)","0(1)","1(2)","6(4)","7(5)","12(3)","13(6)"],"object_location_counts":[{"shards":"0(1),1(2),4(0),6(4),7(5),12(3),13(6)","objects":1}],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"1"}]},{"pgid":"3.a","version":"63'29","reported_seq":212,"reported_epoch":68,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-15T01:33:10.085187+0000","last_change":"2024-09-15T01:33:10.083772+0000","last_active":"2024-09-15T01:33:08.075884+0000","last_peered":"2024-09-15T01:33:07.490619+0000","last_clean":"2024-09-15T01:33:07.490619+0000","last_became_active":"2024-09-15T01:32:57.079821+0000","last_became_peered":"2024-09-15T01:32:57.079821+0000","last_unstale":"2024-09-15T01:33:10.085187+0000","last_undegraded":"2024-09-15T01:33:10.082325+0000","last_fullsized":"2024-09-15T01:33:10.081800+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:57.858851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001893236,"stat_sum":{"num_bytes":5111808,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":3,"num_write":15,"num_write_kb":5584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2359296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,2147483647,14,6,11],"avail_no_missing":["3(0)","1(2)","6(5)","7(1)","8(3)","11(6)","14(4)"],"object_location_counts":[{"shards":"1(2),3(0),6(5),7(1),11(6),14(4)","objects":4}],"blocked_by":[3],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1b","length":"1"}]},{"pgid":"3.9","version":"66'56","reported_seq":169,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.079000+0000","last_change":"2024-09-15T01:33:08.850868+0000","last_active":"2024-09-15T01:33:09.079000+0000","last_peered":"2024-09-15T01:33:09.079000+0000","last_clean":"2024-09-15T01:33:09.079000+0000","last_became_active":"2024-09-15T01:32:34.487117+0000","last_became_peered":"2024-09-15T01:32:34.487117+0000","last_unstale":"2024-09-15T01:33:09.079000+0000","last_undegraded":"2024-09-15T01:33:09.079000+0000","last_fullsized":"2024-09-15T01:33:09.079000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.760814+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.77310157099999999,"stat_sum":{"num_bytes":1277952,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":2602,"num_write":28,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"15","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"3.8","version":"66'65","reported_seq":173,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.076999+0000","last_change":"2024-09-15T01:33:08.081384+0000","last_active":"2024-09-15T01:33:09.076999+0000","last_peered":"2024-09-15T01:33:09.076999+0000","last_clean":"2024-09-15T01:33:09.076999+0000","last_became_active":"2024-09-15T01:32:34.190371+0000","last_became_peered":"2024-09-15T01:32:34.190371+0000","last_unstale":"2024-09-15T01:33:09.076999+0000","last_undegraded":"2024-09-15T01:33:09.076999+0000","last_fullsized":"2024-09-15T01:33:09.076999+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:51.142885+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0064452390000000002,"stat_sum":{"num_bytes":10764288,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":16,"num_read_kb":5433,"num_write":54,"num_write_kb":11200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"15","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"1.0","version":"24'112","reported_seq":188,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.077552+0000","last_change":"2024-09-15T01:32:16.749959+0000","last_active":"2024-09-15T01:33:09.077552+0000","last_peered":"2024-09-15T01:33:09.077552+0000","last_clean":"2024-09-15T01:33:09.077552+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:33:09.077552+0000","last_undegraded":"2024-09-15T01:33:09.077552+0000","last_fullsized":"2024-09-15T01:33:09.077552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_clean_scrub_stamp":"2024-09-15T01:32:15.451214+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":55,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.076725+0000","last_change":"2024-09-15T01:32:25.147954+0000","last_active":"2024-09-15T01:33:09.076725+0000","last_peered":"2024-09-15T01:33:09.076725+0000","last_clean":"2024-09-15T01:33:09.076725+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:33:09.076725+0000","last_undegraded":"2024-09-15T01:33:09.076725+0000","last_fullsized":"2024-09-15T01:33:09.076725+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.228776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"66'36","reported_seq":185,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.076754+0000","last_change":"2024-09-15T01:33:08.091074+0000","last_active":"2024-09-15T01:33:09.076754+0000","last_peered":"2024-09-15T01:33:09.076754+0000","last_clean":"2024-09-15T01:33:09.076754+0000","last_became_active":"2024-09-15T01:32:57.985433+0000","last_became_peered":"2024-09-15T01:32:57.985433+0000","last_unstale":"2024-09-15T01:33:09.076754+0000","last_undegraded":"2024-09-15T01:33:09.076754+0000","last_fullsized":"2024-09-15T01:33:09.076754+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:55.388361+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012259954999999999,"stat_sum":{"num_bytes":1622016,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":3846,"num_write":19,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":5652480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.076900+0000","last_change":"2024-09-15T01:32:25.143709+0000","last_active":"2024-09-15T01:33:09.076900+0000","last_peered":"2024-09-15T01:33:09.076900+0000","last_clean":"2024-09-15T01:33:09.076900+0000","last_became_active":"2024-09-15T01:32:23.134382+0000","last_became_peered":"2024-09-15T01:32:23.134382+0000","last_unstale":"2024-09-15T01:33:09.076900+0000","last_undegraded":"2024-09-15T01:33:09.076900+0000","last_fullsized":"2024-09-15T01:33:09.076900+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.326224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"62'14","reported_seq":200,"reported_epoch":68,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-15T01:33:10.087103+0000","last_change":"2024-09-15T01:33:10.083463+0000","last_active":"2024-09-15T01:33:08.075551+0000","last_peered":"2024-09-15T01:33:07.070424+0000","last_clean":"2024-09-15T01:33:07.070424+0000","last_became_active":"2024-09-15T01:32:58.074255+0000","last_became_peered":"2024-09-15T01:32:58.074255+0000","last_unstale":"2024-09-15T01:33:10.087103+0000","last_undegraded":"2024-09-15T01:33:10.082060+0000","last_fullsized":"2024-09-15T01:33:10.081464+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:59.645939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034743399999999998,"stat_sum":{"num_bytes":1867776,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":1059,"num_write":10,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2129920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,2,13],"acting":[4,11,2147483647,5,10,2,13],"avail_no_missing":["4(0)","2(5)","5(3)","10(4)","11(1)","13(6)","15(2)"],"object_location_counts":[{"shards":"2(5),4(0),5(3),10(4),11(1),13(6),15(2)","objects":3}],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.076571+0000","last_change":"2024-09-15T01:32:25.583876+0000","last_active":"2024-09-15T01:33:09.076571+0000","last_peered":"2024-09-15T01:33:09.076571+0000","last_clean":"2024-09-15T01:33:09.076571+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:33:09.076571+0000","last_undegraded":"2024-09-15T01:33:09.076571+0000","last_fullsized":"2024-09-15T01:33:09.076571+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:49.092006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"56'39","reported_seq":155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.079101+0000","last_change":"2024-09-15T01:33:08.078117+0000","last_active":"2024-09-15T01:33:09.079101+0000","last_peered":"2024-09-15T01:33:09.079101+0000","last_clean":"2024-09-15T01:33:09.079101+0000","last_became_active":"2024-09-15T01:32:34.190384+0000","last_became_peered":"2024-09-15T01:32:34.190384+0000","last_unstale":"2024-09-15T01:33:09.079101+0000","last_undegraded":"2024-09-15T01:33:09.079101+0000","last_fullsized":"2024-09-15T01:33:09.079101+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.588055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069766700000000001,"stat_sum":{"num_bytes":819200,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3144,"num_write":16,"num_write_kb":2832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"15","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":57,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.077048+0000","last_change":"2024-09-15T01:32:25.144113+0000","last_active":"2024-09-15T01:33:09.077048+0000","last_peered":"2024-09-15T01:33:09.077048+0000","last_clean":"2024-09-15T01:33:09.077048+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:33:09.077048+0000","last_undegraded":"2024-09-15T01:33:09.077048+0000","last_fullsized":"2024-09-15T01:33:09.077048+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"66'53","reported_seq":165,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.077261+0000","last_change":"2024-09-15T01:33:08.091133+0000","last_active":"2024-09-15T01:33:09.077261+0000","last_peered":"2024-09-15T01:33:09.077261+0000","last_clean":"2024-09-15T01:33:09.077261+0000","last_became_active":"2024-09-15T01:32:34.170218+0000","last_became_peered":"2024-09-15T01:32:34.170218+0000","last_unstale":"2024-09-15T01:33:09.077261+0000","last_undegraded":"2024-09-15T01:33:09.077261+0000","last_fullsized":"2024-09-15T01:33:09.077261+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.093067+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.011453891000000001,"stat_sum":{"num_bytes":1966080,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":1545,"num_write":45,"num_write_kb":5264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"3.5","version":"66'39","reported_seq":150,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T01:33:08.092465+0000","last_change":"2024-09-15T01:33:08.092465+0000","last_active":"2024-09-15T01:33:08.092465+0000","last_peered":"2024-09-15T01:33:08.092465+0000","last_clean":"2024-09-15T01:33:08.092465+0000","last_became_active":"2024-09-15T01:32:34.189813+0000","last_became_peered":"2024-09-15T01:32:34.189813+0000","last_unstale":"2024-09-15T01:33:08.092465+0000","last_undegraded":"2024-09-15T01:33:08.092465+0000","last_fullsized":"2024-09-15T01:33:08.092465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:47.814751+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.012121722,"stat_sum":{"num_bytes":3981312,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":6008,"num_write":20,"num_write_kb":5712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"15","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.078752+0000","last_change":"2024-09-15T01:32:25.568260+0000","last_active":"2024-09-15T01:33:09.078752+0000","last_peered":"2024-09-15T01:33:09.078752+0000","last_clean":"2024-09-15T01:33:09.078752+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:33:09.078752+0000","last_undegraded":"2024-09-15T01:33:09.078752+0000","last_fullsized":"2024-09-15T01:33:09.078752+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:32.165806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"66'42","reported_seq":151,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.080189+0000","last_change":"2024-09-15T01:33:08.093194+0000","last_active":"2024-09-15T01:33:09.080189+0000","last_peered":"2024-09-15T01:33:09.080189+0000","last_clean":"2024-09-15T01:33:09.080189+0000","last_became_active":"2024-09-15T01:32:34.189644+0000","last_became_peered":"2024-09-15T01:32:34.189644+0000","last_unstale":"2024-09-15T01:33:09.080189+0000","last_undegraded":"2024-09-15T01:33:09.080189+0000","last_fullsized":"2024-09-15T01:33:09.080189+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:02.903663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014535107,"stat_sum":{"num_bytes":4276224,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":21,"num_read_kb":6345,"num_write":41,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"15","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.076260+0000","last_change":"2024-09-15T01:32:25.407396+0000","last_active":"2024-09-15T01:33:09.076260+0000","last_peered":"2024-09-15T01:33:09.076260+0000","last_clean":"2024-09-15T01:33:09.076260+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:33:09.076260+0000","last_undegraded":"2024-09-15T01:33:09.076260+0000","last_fullsized":"2024-09-15T01:33:09.076260+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:39.303667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"57'51","reported_seq":256,"reported_epoch":68,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-15T01:33:10.086794+0000","last_change":"2024-09-15T01:33:10.084055+0000","last_active":"2024-09-15T01:33:08.076182+0000","last_peered":"2024-09-15T01:33:07.070602+0000","last_clean":"2024-09-15T01:33:07.070602+0000","last_became_active":"2024-09-15T01:32:59.541051+0000","last_became_peered":"2024-09-15T01:32:59.541051+0000","last_unstale":"2024-09-15T01:33:10.086794+0000","last_undegraded":"2024-09-15T01:33:10.082699+0000","last_fullsized":"2024-09-15T01:33:10.081854+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:53.849387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025903110000000001,"stat_sum":{"num_bytes":9322496,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":12105,"num_write":26,"num_write_kb":9264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":8208384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,2147483647,0,8,5,11],"avail_no_missing":["4(0)","0(3)","1(2)","5(5)","6(1)","8(4)","11(6)"],"object_location_counts":[{"shards":"0(3),4(0),5(5),6(1),8(4),11(6)","objects":4}],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T01:33:09.081594+0000","last_change":"2024-09-15T01:32:25.568523+0000","last_active":"2024-09-15T01:33:09.081594+0000","last_peered":"2024-09-15T01:33:09.081594+0000","last_clean":"2024-09-15T01:33:09.081594+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:33:09.081594+0000","last_undegraded":"2024-09-15T01:33:09.081594+0000","last_fullsized":"2024-09-15T01:33:09.081594+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"62'36","reported_seq":192,"reported_epoch":68,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-15T01:33:10.087171+0000","last_change":"2024-09-15T01:33:10.085906+0000","last_active":"2024-09-15T01:33:08.075735+0000","last_peered":"2024-09-15T01:33:07.961789+0000","last_clean":"2024-09-15T01:33:07.961789+0000","last_became_active":"2024-09-15T01:32:41.203815+0000","last_became_peered":"2024-09-15T01:32:41.203815+0000","last_unstale":"2024-09-15T01:33:10.087171+0000","last_undegraded":"2024-09-15T01:33:10.084604+0000","last_fullsized":"2024-09-15T01:33:10.083931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:57.288063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051224899999999999,"stat_sum":{"num_bytes":3997696,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":24,"num_read_kb":6812,"num_write":39,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,2147483647],"avail_no_missing":["0(0)","1(2)","7(5)","10(6)","12(4)","13(1)","15(3)"],"object_location_counts":[{"shards":"0(0),1(2),7(5),10(6),12(4),13(1),15(3)","objects":4}],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T01:33:08.078220+0000","last_change":"2024-09-15T01:32:25.582726+0000","last_active":"2024-09-15T01:33:08.078220+0000","last_peered":"2024-09-15T01:33:08.078220+0000","last_clean":"2024-09-15T01:33:08.078220+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:33:08.078220+0000","last_undegraded":"2024-09-15T01:33:08.078220+0000","last_fullsized":"2024-09-15T01:33:08.078220+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.907962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":74973184,"num_objects":55,"num_object_clones":14,"num_object_copies":385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":1,"num_read":273,"num_read_kb":50080,"num_write":512,"num_write_kb":104944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":20021248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":717,"ondisk_log_size":717,"up":112,"acting":112,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313231,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":815148,"kb_used_data":92865188,"kb_used_omap":1542,"kb_used_meta":0,"kb_avail":92864468,"statfs":{"total":95927926784,"available":95093215232,"internally_reserved":737280,"allocated":95093952512,"data_stored":95093952512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1579185,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":3,"commit_latency_ns":0,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":9,"up_from":21,"seq":90194313229,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796892,"kb_used_data":92882792,"kb_used_omap":194,"kb_used_meta":0,"kb_avail":92882724,"statfs":{"total":95927926784,"available":95111909376,"internally_reserved":69632,"allocated":95111979008,"data_stored":95111979008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":199206,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]}]},{"osd":3,"up_from":21,"seq":90194313230,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":802160,"kb_used_data":92878080,"kb_used_omap":775,"kb_used_meta":0,"kb_avail":92877456,"statfs":{"total":95927926784,"available":95106514944,"internally_reserved":638976,"allocated":95107153920,"data_stored":95107153920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":794275,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":5,"commit_latency_ns":0,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]}]},{"osd":0,"up_from":21,"seq":90194313231,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":822564,"kb_used_data":92857784,"kb_used_omap":2051,"kb_used_meta":0,"kb_avail":92857052,"statfs":{"total":95927926784,"available":95085621248,"internally_reserved":749568,"allocated":95086370816,"data_stored":95086370816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2101073,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]}]},{"osd":13,"up_from":21,"seq":90194313230,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":811312,"kb_used_data":92868480,"kb_used_omap":1714,"kb_used_meta":0,"kb_avail":92868304,"statfs":{"total":95927926784,"available":95097143296,"internally_reserved":180224,"allocated":95097323520,"data_stored":95097323520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1755889,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":342.56099999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]}]},{"osd":2,"up_from":21,"seq":90194313229,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":813800,"kb_used_data":92866024,"kb_used_omap":1498,"kb_used_meta":0,"kb_avail":92865816,"statfs":{"total":95927926784,"available":95094595584,"internally_reserved":212992,"allocated":95094808576,"data_stored":95094808576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1534503,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":110.032}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]},{"osd":15,"up_from":21,"seq":90194313230,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":808724,"kb_used_data":92871048,"kb_used_omap":1449,"kb_used_meta":0,"kb_avail":92870892,"statfs":{"total":95927926784,"available":95099793408,"internally_reserved":159744,"allocated":95099953152,"data_stored":95099953152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1484363,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]},{"osd":4,"up_from":21,"seq":90194313230,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":807908,"kb_used_data":92872364,"kb_used_omap":1051,"kb_used_meta":0,"kb_avail":92871708,"statfs":{"total":95927926784,"available":95100628992,"internally_reserved":671744,"allocated":95101300736,"data_stored":95101300736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1077141,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":5,"up_from":21,"seq":90194313229,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810156,"kb_used_data":92869592,"kb_used_omap":982,"kb_used_meta":0,"kb_avail":92869460,"statfs":{"total":95927926784,"available":95098327040,"internally_reserved":135168,"allocated":95098462208,"data_stored":95098462208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1005616,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]}]},{"osd":6,"up_from":21,"seq":90194313229,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":812432,"kb_used_data":92867380,"kb_used_omap":1155,"kb_used_meta":0,"kb_avail":92867184,"statfs":{"total":95927926784,"available":95095996416,"internally_reserved":200704,"allocated":95096197120,"data_stored":95096197120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1183509,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":7,"up_from":21,"seq":90194313230,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":805204,"kb_used_data":92875032,"kb_used_omap":1170,"kb_used_meta":0,"kb_avail":92874412,"statfs":{"total":95927926784,"available":95103397888,"internally_reserved":634880,"allocated":95104032768,"data_stored":95104032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1198481,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":3,"commit_latency_ns":0,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":62.991999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":8,"up_from":21,"seq":90194313229,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":815308,"kb_used_data":92865016,"kb_used_omap":1433,"kb_used_meta":0,"kb_avail":92864308,"statfs":{"total":95927926784,"available":95093051392,"internally_reserved":724992,"allocated":95093776384,"data_stored":95093776384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1467809,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313229,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810800,"kb_used_data":92869500,"kb_used_omap":1480,"kb_used_meta":0,"kb_avail":92868816,"statfs":{"total":95927926784,"available":95097667584,"internally_reserved":700416,"allocated":95098368000,"data_stored":95098368000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1516288,"internal_metadata":0},"hb_peers":[0,1,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":11,"up_from":21,"seq":90194313229,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":809664,"kb_used_data":92870112,"kb_used_omap":968,"kb_used_meta":0,"kb_avail":92869952,"statfs":{"total":95927926784,"available":95098830848,"internally_reserved":163840,"allocated":95098994688,"data_stored":95098994688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":991301,"internal_metadata":0},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":3,"commit_latency_ns":0,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":12,"up_from":21,"seq":90194313230,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":802440,"kb_used_data":92877308,"kb_used_omap":770,"kb_used_meta":0,"kb_avail":92877176,"statfs":{"total":95927926784,"available":95106228224,"internally_reserved":135168,"allocated":95106363392,"data_stored":95106363392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":789415,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]}]},{"osd":1,"up_from":21,"seq":90194313230,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":813564,"kb_used_data":92866272,"kb_used_omap":1575,"kb_used_meta":0,"kb_avail":92866052,"statfs":{"total":95927926784,"available":95094837248,"internally_reserved":225280,"allocated":95095062528,"data_stored":95095062528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1613595,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]}]}],"pool_statfs":[]}} 2024-09-15T01:33:14.032 INFO:tasks.thrashosds.thrasher:Setting 3.9 to [11, 3, 6, 0, 13, 5, 9] 2024-09-15T01:33:14.032 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.9', '11', '3', '6', '0', '13', '5', '9'] 2024-09-15T01:33:14.032 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.9 11 3 6 0 13 5 9 2024-09-15T01:33:14.039 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:14.040 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:14.040 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:14.038+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:14.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:14.044+0000 7f3a81d9b640 0 -- 172.21.15.42:0/2979180777 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7f3a84191a40 msgr2=0x7f3a84193ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:14.065 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:14.066 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:14.085 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:14.085 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:14.087 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:14.088 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:14.096 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:33:14.103 INFO:tasks.rados.rados.0.smithi175.stdout:276: done (2 left) 2024-09-15T01:33:14.103 INFO:tasks.rados.rados.0.smithi175.stdout:277: done (1 left) 2024-09-15T01:33:14.103 INFO:tasks.rados.rados.0.smithi175.stdout:278: copy_from oid 13 from oid 5 current snap is 21 2024-09-15T01:33:14.103 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 16 current snap is 21 2024-09-15T01:33:14.103 INFO:tasks.rados.rados.0.smithi175.stdout:279: seq_num 130 ranges {1949696=638976,2588672=671744,3260416=802816,4063232=81920} 2024-09-15T01:33:14.117 INFO:tasks.rados.rados.0.smithi175.stdout:279: writing smithi17529310-16 from 1949696 to 2588672 tid 1 2024-09-15T01:33:14.122 INFO:tasks.rados.rados.0.smithi175.stdout:279: writing smithi17529310-16 from 2588672 to 3260416 tid 2 2024-09-15T01:33:14.127 INFO:tasks.rados.rados.0.smithi175.stdout:279: writing smithi17529310-16 from 3260416 to 4063232 tid 3 2024-09-15T01:33:14.127 INFO:tasks.rados.rados.0.smithi175.stdout:279: writing smithi17529310-16 from 4063232 to 4145152 tid 4 2024-09-15T01:33:14.128 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 36 current snap is 21 2024-09-15T01:33:14.128 INFO:tasks.rados.rados.0.smithi175.stdout:280: seq_num 131 ranges {737280=655360,1392640=425984,1818624=638976,2457600=491520} 2024-09-15T01:33:14.135 INFO:tasks.rados.rados.0.smithi175.stdout:280: writing smithi17529310-36 from 737280 to 1392640 tid 1 2024-09-15T01:33:14.138 INFO:tasks.rados.rados.0.smithi175.stdout:280: writing smithi17529310-36 from 1392640 to 1818624 tid 2 2024-09-15T01:33:14.139 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:14.137+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:14.142 INFO:tasks.rados.rados.0.smithi175.stdout:280: writing smithi17529310-36 from 1818624 to 2457600 tid 3 2024-09-15T01:33:14.145 INFO:tasks.rados.rados.0.smithi175.stdout:280: writing smithi17529310-36 from 2457600 to 2949120 tid 4 2024-09-15T01:33:14.145 INFO:tasks.rados.rados.0.smithi175.stdout:281: snap_remove snap 15 2024-09-15T01:33:14.205 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:14.205 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:14.213 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:33:14.239 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:14.238+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:14.340 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:14.339+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:14.350 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:14.350 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:14.359 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:33:14.441 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:14.440+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:14.497 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:14.497 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:14.506 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:33:14.543 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:14.541+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:14.604 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:14.604 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:14.613 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:33:14.643 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:14.642+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:14.708 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:14.709 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:14.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:33:14.744 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:14.743+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:14.812 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:14.812 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:14.820 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:33:14.845 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:14.843+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:14.916 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:14.916 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:14.924 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:33:14.967 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:14.966+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:15.020 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:15.020 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:15.028 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:33:15.046 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:15.044+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:15.119 INFO:teuthology.orchestra.run.smithi042.stderr:set 3.9 pg_upmap mapping to [11,3,6,0,13,5,9] 2024-09-15T01:33:15.123 INFO:tasks.rados.rados.0.smithi175.stdout:281: done (4 left) 2024-09-15T01:33:15.123 INFO:tasks.rados.rados.0.smithi175.stdout:282: delete oid 34 current snap is 21 2024-09-15T01:33:15.123 INFO:tasks.rados.rados.0.smithi175.stdout:278: finishing copy_from racing read to smithi17529310-13 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:278: finishing copy_from to smithi17529310-13 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 13 v 8 (ObjNum 95 snap 9 seq_num 95) dirty exists 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:279: finishing write tid 1 to smithi17529310-16 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:279: finishing write tid 2 to smithi17529310-16 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:279: finishing write tid 3 to smithi17529310-16 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:279: finishing write tid 4 to smithi17529310-16 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:279: finishing write tid 5 to smithi17529310-16 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:279: finishing write tid 6 to smithi17529310-16 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 16 v 126 (ObjNum 130 snap 21 seq_num 130) dirty exists 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:279: left oid 16 (ObjNum 130 snap 21 seq_num 130) 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:280: finishing write tid 1 to smithi17529310-36 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:280: finishing write tid 2 to smithi17529310-36 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:280: finishing write tid 3 to smithi17529310-36 2024-09-15T01:33:15.124 INFO:tasks.rados.rados.0.smithi175.stdout:280: finishing write tid 4 to smithi17529310-36 2024-09-15T01:33:15.125 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 2 v 137 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-15T01:33:15.125 INFO:tasks.rados.rados.0.smithi175.stdout:280: finishing write tid 5 to smithi17529310-36 2024-09-15T01:33:15.125 INFO:tasks.rados.rados.0.smithi175.stdout:280: finishing write tid 6 to smithi17529310-36 2024-09-15T01:33:15.125 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 36 v 67 (ObjNum 131 snap 21 seq_num 131) dirty exists 2024-09-15T01:33:15.125 INFO:tasks.rados.rados.0.smithi175.stdout:280: left oid 36 (ObjNum 131 snap 21 seq_num 131) 2024-09-15T01:33:15.133 INFO:tasks.rados.rados.0.smithi175.stdout:273: done (4 left) 2024-09-15T01:33:15.133 INFO:tasks.rados.rados.0.smithi175.stdout:278: done (3 left) 2024-09-15T01:33:15.133 INFO:tasks.rados.rados.0.smithi175.stdout:279: done (2 left) 2024-09-15T01:33:15.133 INFO:tasks.rados.rados.0.smithi175.stdout:280: done (1 left) 2024-09-15T01:33:15.133 INFO:tasks.rados.rados.0.smithi175.stdout:282: done (0 left) 2024-09-15T01:33:15.133 INFO:tasks.rados.rados.0.smithi175.stdout:283: rollback oid 48 current snap is 21 2024-09-15T01:33:15.133 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 48 to 16 2024-09-15T01:33:15.133 INFO:tasks.rados.rados.0.smithi175.stdout:284: copy_from oid 28 from oid 41 current snap is 21 2024-09-15T01:33:15.134 INFO:tasks.rados.rados.0.smithi175.stdout:285: snap_create 2024-09-15T01:33:15.138 INFO:tasks.rados.rados.0.smithi175.stdout:283: finishing rollback tid 0 to smithi17529310-48 2024-09-15T01:33:15.138 INFO:tasks.rados.rados.0.smithi175.stdout:283: finishing rollback tid 1 to smithi17529310-48 2024-09-15T01:33:15.138 INFO:tasks.rados.rados.0.smithi175.stdout:283: finishing rollback tid 2 to smithi17529310-48 2024-09-15T01:33:15.138 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 48 v 83 (ObjNum 84 snap 7 seq_num 84) dirty exists 2024-09-15T01:33:15.139 INFO:tasks.rados.rados.0.smithi175.stdout:284: finishing copy_from to smithi17529310-28 2024-09-15T01:33:15.140 INFO:tasks.rados.rados.0.smithi175.stdout:284: got expected ENOENT (src dne) 2024-09-15T01:33:15.140 INFO:tasks.rados.rados.0.smithi175.stdout:284: finishing copy_from racing read to smithi17529310-28 2024-09-15T01:33:15.144 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:33:15.145 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:15.144+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:15.176 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:15.176 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:15.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-15T01:33:15.246 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:15.245+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:15.321 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:15.322 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:15.330 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-15T01:33:15.347 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:15.345+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:15.426 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:15.426 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:15.435 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-15T01:33:15.448 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:15.446+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:15.532 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:15.532 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:15.540 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:33:15.550 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:15.548+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:15.642 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:15.642 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:15.651 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:33:15.651 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:15.649+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:15.751 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:15.749+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:15.751 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:15.751 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:15.759 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:33:15.851 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:15.850+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:15.860 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:15.860 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:15.868 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:33:15.952 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:15.950+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:15.968 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:15.968 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:15.977 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:33:16.053 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:16.050+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-15T01:33:16.073 INFO:tasks.rados.rados.0.smithi175.stdout:283: done (2 left) 2024-09-15T01:33:16.073 INFO:tasks.rados.rados.0.smithi175.stdout:284: done (1 left) 2024-09-15T01:33:16.073 INFO:tasks.rados.rados.0.smithi175.stdout:285: done (0 left) 2024-09-15T01:33:16.073 INFO:tasks.rados.rados.0.smithi175.stdout:286: setattr oid 30 current snap is 22 2024-09-15T01:33:16.076 INFO:tasks.rados.rados.0.smithi175.stdout:287: snap_create 2024-09-15T01:33:16.079 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 30 v 60 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-09-15T01:33:16.082 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:16.082 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:16.091 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:33:16.096 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:33:16.153 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:16.151+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:16.233 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:16.233 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:16.242 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:33:16.254 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:16.251+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:16.258 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:16.277 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:16.278 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:16.297 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:16.297 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:16.317 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:16.317 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:16.335 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:16.344 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:16.344 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:16.353 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:33:16.355 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:16.356 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:16.354+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:16.374 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:16.375 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:16.397 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:16.397 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:16.415 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:16.435 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:16.435 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:16.454 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:16.455 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:16.456 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:16.456 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:33:16.456 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:33:16.456 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:16.456 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:16.454+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:16.464 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:33:16.473 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:16.492 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:16.492 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:16.511 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:16.531 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:16.531 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:16.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:16.529+0000 7fdcf77fe640 0 -- 172.21.15.42:0/3806023327 >> [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] conn(0x7fdcf8197f50 msgr2=0x7fdcf819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-15T01:33:16.556 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:16.555+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:16.577 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.5s1 33/31 ECSubWrite(tid=1, reqid=client.4477.0:11, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.396831986999999, 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030602694, 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8900000000000001e-07 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178311+0000", 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5959999999999993e-06 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208281+0000", 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029970173999999999 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208291+0000", 2024-09-15T01:33:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3850000000000008e-06 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208407+0000", 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116168 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208422+0000", 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4931e-05 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208519+0000", 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7213999999999995e-05 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208541+0000", 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2249999999999999e-05 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208644+0000", 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102537 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208664+0000", 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.071e-05 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208904+0000", 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023944 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.5s1 34/31 ECSubRead(tid=6, to_read={3:af8bec4d:::smithi17529310-24:head=0,4096,0}, subchunks={3:af8bec4d:::smithi17529310-24:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.210749+0000", 2024-09-15T01:33:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.364384111, 2024-09-15T01:33:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033380170000000001, 2024-09-15T01:33:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210749+0000", 2024-09-15T01:33:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210749+0000", 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210750+0000", 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6000000000000004e-07 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210754+0000", 2024-09-15T01:33:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7049999999999998e-06 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.243939+0000", 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033185092999999999 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.243951+0000", 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1864e-05 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.244010+0000", 2024-09-15T01:33:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8536000000000003e-05 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.244129+0000", 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119412 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/31 ECSubWrite(tid=22, reqid=client.4477.0:113, at_version=34'10, trim_to=0'0, roll_forward_to=34'7))", 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:33:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.319914238000003, 2024-09-15T01:33:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035418835000000003, 2024-09-15T01:33:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255233+0000", 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3815000000000001e-05 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.289999+0000", 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.034765375000000001 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290004+0000", 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290062+0000", 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7763999999999998e-05 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290072+0000", 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7829999999999993e-06 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290165+0000", 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3039000000000001e-05 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290192+0000", 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7449000000000001e-05 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290303+0000", 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110253 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290375+0000", 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2855999999999995e-05 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290638+0000", 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026231300000000002 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:128 3.4s0 3:299c4018:::smithi17529310-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.275429+0000", 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.299704136000003, 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.10370515499999999, 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 128 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275429+0000", 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275429+0000", 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275430+0000", 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1099999999999995e-07 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275434+0000", 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.42e-06 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275443+0000", 2024-09-15T01:33:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1330000000000005e-06 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275448+0000", 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3769999999999998e-06 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.276441+0000", 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099363500000000009 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.276580+0000", 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013892099999999999 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277288+0000", 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070808500000000001 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277385+0000", 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6824999999999995e-05 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277394+0000", 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0839999999999998e-06 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277434+0000", 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9465999999999998e-05 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277551+0000", 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117162 2024-09-15T01:33:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379048+0000", 2024-09-15T01:33:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101497254 2024-09-15T01:33:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379134+0000", 2024-09-15T01:33:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6081999999999993e-05 2024-09-15T01:33:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:129 3.4s0 3:299c4018:::smithi17529310-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:33:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.299664294999999, 2024-09-15T01:33:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.105118901, 2024-09-15T01:33:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 129 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8999999999999996e-07 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275472+0000", 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6419999999999999e-06 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275480+0000", 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0790000000000001e-06 2024-09-15T01:33:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275485+0000", 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4689999999999999e-06 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277457+0000", 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019728160000000001 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277534+0000", 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6073000000000002e-05 2024-09-15T01:33:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379464+0000", 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.10193010499999999 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379556+0000", 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2084999999999997e-05 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.380588+0000", 2024-09-15T01:33:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010320419999999999 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.cs1 34/31 ECSubRead(tid=37, to_read={3:34a45b5f:::smithi17529310-48:head=0,4096,0}, subchunks={3:34a45b5f:::smithi17529310-48:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.267298480999997, 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14311833299999999, 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:33:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6499999999999999e-07 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307844+0000", 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4810000000000006e-06 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450796+0000", 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14295194899999999 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450806+0000", 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7890000000000006e-06 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450858+0000", 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2577999999999997e-05 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450953+0000", 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4870999999999993e-05 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:217 3.4s0 3:299c4018:::smithi17529310-43:head [copy-from ver 4] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.208686+0000", 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 35.366447061999999, 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013624864, 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 217 2024-09-15T01:33:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208686+0000", 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208686+0000", 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208688+0000", 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3989999999999998e-06 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208709+0000", 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0849999999999999e-05 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208729+0000", 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9681e-05 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208738+0000", 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5350000000000004e-06 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208841+0000", 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103353 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208941+0000", 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9986000000000001e-05 2024-09-15T01:33:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.219554+0000", 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01061337 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.219922+0000", 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036728800000000001 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.219935+0000", 2024-09-15T01:33:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3689000000000001e-05 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220006+0000", 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1073000000000002e-05 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220034+0000", 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7736e-05 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.221399+0000", 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013649000000000001 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.222311+0000", 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000912004 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:218 3.4s0 3:299c4018:::smithi17529310-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:33:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 35.366380526, 2024-09-15T01:33:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011546681, 2024-09-15T01:33:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 218 2024-09-15T01:33:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6799999999999999e-07 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208757+0000", 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.941e-06 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208767+0000", 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8509999999999994e-06 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208786+0000", 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8320000000000001e-05 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209130+0000", 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034496100000000002 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209184+0000", 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3703999999999999e-05 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220077+0000", 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010893277999999999 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220195+0000", 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117342 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220299+0000", 2024-09-15T01:33:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104516 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs1 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.241172+0000", 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.333960810000001, 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018176115, 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241172+0000", 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241172+0000", 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241173+0000", 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.05e-07 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241195+0000", 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2033000000000001e-05 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241238+0000", 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3507999999999997e-05 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241245+0000", 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2509999999999997e-06 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245590+0000", 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043445309999999996 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245599+0000", 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8519999999999993e-06 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246349+0000", 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00075022999999999995 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246362+0000", 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.273e-05 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246399+0000", 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7299000000000003e-05 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246806+0000", 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040714800000000003 2024-09-15T01:33:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.259348+0000", 2024-09-15T01:33:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012542328 2024-09-15T01:33:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs1 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:33:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.331867568, 2024-09-15T01:33:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017149162999999999, 2024-09-15T01:33:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:33:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243294+0000", 2024-09-15T01:33:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7868000000000002e-05 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243334+0000", 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9369000000000002e-05 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243341+0000", 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7009999999999996e-06 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246522+0000", 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031802359999999999 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246530+0000", 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5019999999999992e-06 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246730+0000", 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019988499999999999 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246745+0000", 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4632e-05 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246784+0000", 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9072000000000001e-05 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246850+0000", 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699000000000006e-05 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.260415+0000", 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013564296 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds5 44/31 ECSubWrite(tid=36, reqid=osd.7.0:36, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.329142567000002, 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013253133, 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:33:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6199999999999999e-07 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246013+0000", 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1801e-05 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246123+0000", 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109879 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246132+0000", 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8670000000000006e-06 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246240+0000", 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107211 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246248+0000", 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8480000000000007e-06 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246452+0000", 2024-09-15T01:33:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020319399999999999 2024-09-15T01:33:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246461+0000", 2024-09-15T01:33:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-15T01:33:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246492+0000", 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1124999999999998e-05 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246831+0000", 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033878400000000001 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.259244+0000", 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012412444 2024-09-15T01:33:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.246909+0000", 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.328224539000001, 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013946906, 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246909+0000", 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246909+0000", 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246910+0000", 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.16e-07 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246933+0000", 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3884e-05 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246981+0000", 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7194000000000002e-05 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246988+0000", 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0260000000000002e-06 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247055+0000", 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7216000000000002e-05 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247066+0000", 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1562999999999999e-05 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248672+0000", 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001605455 2024-09-15T01:33:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248697+0000", 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5588e-05 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248730+0000", 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2351000000000002e-05 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248812+0000", 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2254000000000002e-05 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.260856+0000", 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012043458999999999 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds5 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:33:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.328006651999999, 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014066664, 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:33:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4000000000000001e-07 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247148+0000", 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1339e-05 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247188+0000", 2024-09-15T01:33:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9301000000000002e-05 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247196+0000", 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2789999999999998e-06 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247453+0000", 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025692899999999998 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247463+0000", 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0317e-05 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248725+0000", 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012613730000000001 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248732+0000", 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9129999999999997e-06 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248776+0000", 2024-09-15T01:33:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4280000000000003e-05 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248811+0000", 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5553000000000003e-05 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.261193+0000", 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01238174 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as3 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.327273400999999, 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012019658000000001, 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:33:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247861+0000", 2024-09-15T01:33:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6700000000000002e-07 2024-09-15T01:33:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247872+0000", 2024-09-15T01:33:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1307e-05 2024-09-15T01:33:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247909+0000", 2024-09-15T01:33:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7290999999999999e-05 2024-09-15T01:33:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247917+0000", 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3590000000000002e-06 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248032+0000", 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115566 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248048+0000", 2024-09-15T01:33:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6069000000000001e-05 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248184+0000", 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013525399999999999 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248205+0000", 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1021000000000001e-05 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248244+0000", 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9847e-05 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248260+0000", 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5639000000000001e-05 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.259879+0000", 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011619338 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.654 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:292 3.4s0 3:299c4018:::smithi17529310-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.267708+0000", 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.307425260999999, 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020077541000000001, 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:16.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 292 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267708+0000", 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267708+0000", 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267709+0000", 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7800000000000001e-07 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267712+0000", 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267723+0000", 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1388e-05 2024-09-15T01:33:16.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267729+0000", 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5890000000000002e-06 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269684+0000", 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019553389999999999 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269888+0000", 2024-09-15T01:33:16.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020362599999999999 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.273250+0000", 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033616280000000002 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.273338+0000", 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7990000000000003e-05 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.287785+0000", 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01444779 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:598 3.4s0 3:269bd1ac:::smithi17529310-23:head [append 0~753664] snapc 1e=[1e,1d,1a,19,15] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:06.076859+0000", 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.498274073999999, 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012447475, 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 598 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.076859+0000", 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.076859+0000", 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.076861+0000", 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.077713+0000", 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085228799999999998 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.077727+0000", 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3472000000000001e-05 2024-09-15T01:33:16.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.077732+0000", 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4299999999999997e-06 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.077778+0000", 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6081000000000001e-05 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.077783+0000", 2024-09-15T01:33:16.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9949999999999996e-06 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.078931+0000", 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011474409999999999 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.079059+0000", 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012866 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.080253+0000", 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011939310000000001 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.080539+0000", 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028544900000000001 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.080551+0000", 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2418e-05 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.080706+0000", 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015443999999999999 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.080727+0000", 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1821e-05 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.089262+0000", 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085342019999999994 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.089307+0000", 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5006999999999998e-05 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs1 66/31 ECSubWrite(tid=198, reqid=osd.11.0:198, at_version=66'62, trim_to=0'0, roll_forward_to=59'58))", 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:08.082679+0000", 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4924540999999998, 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010665167, 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:16.667 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082679+0000", 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082679+0000", 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082679+0000", 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4499999999999998e-07 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082694+0000", 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4242e-05 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082914+0000", 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022094500000000001 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082920+0000", 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084323+0000", 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001403191 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084331+0000", 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3450000000000006e-06 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084767+0000", 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043536399999999999 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084784+0000", 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6939000000000002e-05 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084820+0000", 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6025000000000002e-05 2024-09-15T01:33:16.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084884+0000", 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3848000000000003e-05 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.093344+0000", 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084606630000000002 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:642 3.4s0 3:269bd1ac:::smithi17529310-23:head [read 0~2293760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:11.515875+0000", 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.059258196, 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.97750444800000003, 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 642 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.515875+0000", 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.515875+0000", 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.515875+0000", 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7599999999999997e-07 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.515878+0000", 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0149999999999999e-06 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.515885+0000", 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7660000000000001e-06 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.515889+0000", 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8330000000000004e-06 2024-09-15T01:33:16.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.517051+0000", 2024-09-15T01:33:16.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011621019999999999 2024-09-15T01:33:16.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:16.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.517251+0000", 2024-09-15T01:33:16.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000199617 2024-09-15T01:33:16.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.493379+0000", 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.97612863900000002 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:666 3.4s0 3:299c4018:::smithi17529310-43:head [append 0~524288] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.238057+0000", 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.337076465, 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010740744, 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.677 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 666 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238057+0000", 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238057+0000", 2024-09-15T01:33:16.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238059+0000", 2024-09-15T01:33:16.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9460000000000001e-06 2024-09-15T01:33:16.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238633+0000", 2024-09-15T01:33:16.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00057398900000000001 2024-09-15T01:33:16.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238654+0000", 2024-09-15T01:33:16.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1778000000000001e-05 2024-09-15T01:33:16.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238661+0000", 2024-09-15T01:33:16.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4999999999999996e-06 2024-09-15T01:33:16.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238725+0000", 2024-09-15T01:33:16.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3669000000000003e-05 2024-09-15T01:33:16.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:16.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238866+0000", 2024-09-15T01:33:16.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014093299999999999 2024-09-15T01:33:16.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.239996+0000", 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011303800000000001 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.240351+0000", 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035475600000000002 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.240365+0000", 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4279e-05 2024-09-15T01:33:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.240542+0000", 2024-09-15T01:33:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017720600000000001 2024-09-15T01:33:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.240564+0000", 2024-09-15T01:33:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1753999999999999e-05 2024-09-15T01:33:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.248736+0000", 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081716800000000006 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.248797+0000", 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1873999999999997e-05 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:671 3.4s0 3:299c4018:::smithi17529310-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.249068+0000", 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.326064841, 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012181770999999999, 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 671 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.249068+0000", 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.249068+0000", 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.249069+0000", 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.72e-07 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.249070+0000", 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1990000000000001e-06 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.249073+0000", 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4690000000000002e-06 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.249078+0000", 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6779999999999999e-06 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.253529+0000", 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044515609999999997 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.253631+0000", 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101222 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.261250+0000", 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0076193700000000003 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:16.689 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:16.689 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:16.690 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-15T01:33:16.691 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:16.688+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:16.752 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:16.753 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:16.791 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:16.788+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:16.834 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:33:16.834 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:16.842 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-15T01:33:16.891 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:16.889+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:16.942 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:16.943 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:16.943 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:33:16.943 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:33:16.943 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:16.951 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-15T01:33:16.992 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:16.990+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:17.053 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:33:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:33:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:33:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 34/31 ECSubWrite(tid=7, reqid=client.4477.0:44, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.861963359000001, 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.119995086, 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188439+0000", 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5613000000000001e-05 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307929+0000", 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.119490443 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307936+0000", 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1740000000000003e-06 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308009+0000", 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2714999999999998e-05 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308018+0000", 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8019999999999998e-06 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308149+0000", 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130739 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308161+0000", 2024-09-15T01:33:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2481e-05 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308199+0000", 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7492e-05 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308216+0000", 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7076999999999999e-05 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308418+0000", 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020204899999999999 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 39/38 ECSubWrite(tid=11, reqid=client.4477.0:196, at_version=39'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 35.844078144000001, 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042772959999999999, 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8800000000000002e-07 2024-09-15T01:33:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206341+0000", 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2418e-05 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208467+0000", 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021264159999999999 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208477+0000", 2024-09-15T01:33:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6090000000000007e-06 2024-09-15T01:33:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208544+0000", 2024-09-15T01:33:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6804000000000006e-05 2024-09-15T01:33:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208555+0000", 2024-09-15T01:33:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0869e-05 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208869+0000", 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031429800000000002 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208897+0000", 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8416999999999999e-05 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208959+0000", 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2201999999999998e-05 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208984+0000", 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4618999999999999e-05 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210585+0000", 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001601056 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.064 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 39/38 ECSubWrite(tid=12, reqid=client.4477.0:197, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.208410+0000", 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 35.841975699999999, 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042832929999999996, 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208410+0000", 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208410+0000", 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208411+0000", 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7100000000000011e-07 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208438+0000", 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6424e-05 2024-09-15T01:33:17.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209653+0000", 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001215617 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209664+0000", 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0244999999999999e-05 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209722+0000", 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8675000000000003e-05 2024-09-15T01:33:17.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209732+0000", 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3470000000000005e-06 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209848+0000", 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011628099999999999 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209861+0000", 2024-09-15T01:33:17.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.359e-05 2024-09-15T01:33:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209921+0000", 2024-09-15T01:33:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9296000000000003e-05 2024-09-15T01:33:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210211+0000", 2024-09-15T01:33:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029074999999999999 2024-09-15T01:33:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212694+0000", 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024820969999999999 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 39/38 ECSubWrite(tid=13, reqid=client.4477.0:198, at_version=39'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 35.840811469000002, 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032744699999999998, 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.070 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:33:17.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2999999999999999e-07 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209623+0000", 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7089999999999998e-05 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211317+0000", 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016948569999999999 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211325+0000", 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9699999999999999e-06 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211380+0000", 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4938000000000002e-05 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211389+0000", 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7439999999999996e-06 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211510+0000", 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012121 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211545+0000", 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5209999999999997e-05 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211601+0000", 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5541000000000002e-05 2024-09-15T01:33:17.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211625+0000", 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4334000000000001e-05 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212849+0000", 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012236459999999999 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.241466+0000", 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.808919563, 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080416059999999998, 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241466+0000", 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241466+0000", 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241467+0000", 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3999999999999998e-07 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241497+0000", 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0596999999999999e-05 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243396+0000", 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018991819999999999 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243404+0000", 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2119999999999997e-06 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243529+0000", 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012554800000000001 2024-09-15T01:33:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243541+0000", 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1834000000000001e-05 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243948+0000", 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040691300000000001 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244199+0000", 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025096899999999998 2024-09-15T01:33:17.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244271+0000", 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1666000000000001e-05 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245015+0000", 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00074422900000000001 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249508+0000", 2024-09-15T01:33:17.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044932160000000004 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.807898005999999, 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056362690000000002, 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0699999999999998e-07 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242511+0000", 2024-09-15T01:33:17.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2926999999999999e-05 2024-09-15T01:33:17.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242662+0000", 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015114499999999999 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242671+0000", 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7730000000000005e-06 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242736+0000", 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4651000000000002e-05 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242746+0000", 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0203e-05 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242974+0000", 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022804800000000001 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243002+0000", 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7792000000000001e-05 2024-09-15T01:33:17.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243044+0000", 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2657999999999997e-05 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243365+0000", 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032055899999999999 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248124+0000", 2024-09-15T01:33:17.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047592060000000002 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242514+0000", 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.807871742, 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090428599999999998, 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242514+0000", 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242514+0000", 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242515+0000", 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242548+0000", 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3615000000000003e-05 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242692+0000", 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014390900000000001 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242698+0000", 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4160000000000003e-06 2024-09-15T01:33:17.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243566+0000", 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086795700000000004 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243574+0000", 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6889999999999993e-06 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243798+0000", 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000223611 2024-09-15T01:33:17.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244198+0000", 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040037200000000002 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244270+0000", 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1956e-05 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244333+0000", 2024-09-15T01:33:17.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2889e-05 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251557+0000", 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072240569999999999 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 44/38 ECSubWrite(tid=20, reqid=osd.0.0:20, at_version=44'12, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.807819883000001, 2024-09-15T01:33:17.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052822889999999999, 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4600000000000001e-07 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242585+0000", 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9040000000000001e-05 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243433+0000", 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084765999999999997 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243441+0000", 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4179999999999997e-06 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243507+0000", 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5222999999999995e-05 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243517+0000", 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0567000000000001e-05 2024-09-15T01:33:17.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243744+0000", 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022733200000000001 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244198+0000", 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045302899999999998 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244269+0000", 2024-09-15T01:33:17.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1445999999999993e-05 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244295+0000", 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5799000000000002e-05 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247848+0000", 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003553629 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243616+0000", 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.806769575000001, 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081799560000000004, 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243616+0000", 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243616+0000", 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243617+0000", 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243646+0000", 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.879e-05 2024-09-15T01:33:17.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243680+0000", 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4082000000000003e-05 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243686+0000", 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1589999999999998e-06 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243983+0000", 2024-09-15T01:33:17.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029721700000000001 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243991+0000", 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6240000000000002e-06 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244172+0000", 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018103300000000001 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244199+0000", 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7254999999999999e-05 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244271+0000", 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.203e-05 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245044+0000", 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077298299999999998 2024-09-15T01:33:17.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251796+0000", 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006752096 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.247492+0000", 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.802894166000002, 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045361799999999999, 2024-09-15T01:33:17.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247492+0000", 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247492+0000", 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247494+0000", 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7230000000000001e-06 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247507+0000", 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3421000000000001e-05 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247558+0000", 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0801999999999999e-05 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247567+0000", 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9889999999999998e-06 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247635+0000", 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8623999999999999e-05 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247647+0000", 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1844999999999999e-05 2024-09-15T01:33:17.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247812+0000", 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016474600000000001 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247865+0000", 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3183000000000002e-05 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247909+0000", 2024-09-15T01:33:17.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3645e-05 2024-09-15T01:33:17.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247928+0000", 2024-09-15T01:33:17.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9049e-05 2024-09-15T01:33:17.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252028+0000", 2024-09-15T01:33:17.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041001529999999996 2024-09-15T01:33:17.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.108 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 44/38 ECSubWrite(tid=21, reqid=osd.0.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.80216021, 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040969320000000002, 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248235+0000", 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7180000000000002e-06 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248464+0000", 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022932900000000001 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248473+0000", 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.789e-06 2024-09-15T01:33:17.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248516+0000", 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3380999999999997e-05 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248526+0000", 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2969999999999992e-06 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248624+0000", 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8672000000000007e-05 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248645+0000", 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0327e-05 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248697+0000", 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2602000000000002e-05 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248930+0000", 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023236900000000001 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252323+0000", 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003393063 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 55/31 ECSubWrite(tid=159, reqid=client.4477.0:511, at_version=55'52, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:17.114 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.075075+0000", 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.975310654000001, 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080575170000000002, 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075075+0000", 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075075+0000", 2024-09-15T01:33:17.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075076+0000", 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9999999999999999e-07 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075102+0000", 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6071000000000002e-05 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075431+0000", 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032964000000000002 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075438+0000", 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8349999999999998e-06 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075493+0000", 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5220999999999999e-05 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075504+0000", 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0434999999999999e-05 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075781+0000", 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027726299999999998 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075914+0000", 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132733 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075957+0000", 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3195999999999998e-05 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076225+0000", 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026822800000000002 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.083133+0000", 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069075949999999999 2024-09-15T01:33:17.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.120 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.120 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 55/31 ECSubWrite(tid=160, reqid=client.4477.0:512, at_version=55'53, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:17.120 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.075980+0000", 2024-09-15T01:33:17.120 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.974406025, 2024-09-15T01:33:17.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011476257, 2024-09-15T01:33:17.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.120 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075980+0000", 2024-09-15T01:33:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075980+0000", 2024-09-15T01:33:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075981+0000", 2024-09-15T01:33:17.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.54e-07 2024-09-15T01:33:17.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075999+0000", 2024-09-15T01:33:17.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7974000000000001e-05 2024-09-15T01:33:17.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076029+0000", 2024-09-15T01:33:17.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0642000000000002e-05 2024-09-15T01:33:17.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076035+0000", 2024-09-15T01:33:17.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9750000000000004e-06 2024-09-15T01:33:17.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076082+0000", 2024-09-15T01:33:17.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6227999999999999e-05 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076091+0000", 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2399999999999996e-06 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076185+0000", 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4315999999999996e-05 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076211+0000", 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5728000000000002e-05 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076263+0000", 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1900000000000001e-05 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076277+0000", 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4239999999999999e-05 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.087456+0000", 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01117916 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 55/31 ECSubWrite(tid=161, reqid=client.4477.0:513, at_version=55'54, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.076704+0000", 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.973682402000001, 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010911452, 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076704+0000", 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076704+0000", 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076704+0000", 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076723+0000", 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8493000000000001e-05 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076754+0000", 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1479999999999997e-05 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076760+0000", 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.105e-06 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076815+0000", 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5061e-05 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076824+0000", 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7110000000000001e-06 2024-09-15T01:33:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076921+0000", 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7176999999999995e-05 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076950+0000", 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8606999999999999e-05 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.077617+0000", 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00066715999999999998 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.077636+0000", 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8779000000000001e-05 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.087615+0000", 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0099791570000000007 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 55/31 ECSubWrite(tid=162, reqid=client.4477.0:513, at_version=0'0, trim_to=0'0, roll_forward_to=55'54))", 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.078911+0000", 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.971475224999999, 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089080270000000007, 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078911+0000", 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078911+0000", 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078912+0000", 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.088e-06 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078924+0000", 2024-09-15T01:33:17.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2592000000000001e-05 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078964+0000", 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9545999999999997e-05 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078973+0000", 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7490000000000004e-06 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079047+0000", 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3755000000000001e-05 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079057+0000", 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0383e-05 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079195+0000", 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013842999999999999 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079269+0000", 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3774000000000002e-05 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079488+0000", 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021939400000000001 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079506+0000", 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7153999999999999e-05 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.087819+0000", 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083131620000000007 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 66/31 ECSubWrite(tid=198, reqid=osd.11.0:198, at_version=66'62, trim_to=0'0, roll_forward_to=59'58))", 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:08.080233+0000", 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.9701532470000007, 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040668559999999998, 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.137 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.080233+0000", 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.080233+0000", 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.080233+0000", 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8599999999999999e-07 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.080254+0000", 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605e-05 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.080362+0000", 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010848 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.080367+0000", 2024-09-15T01:33:17.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1540000000000001e-06 2024-09-15T01:33:17.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082282+0000", 2024-09-15T01:33:17.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001915157 2024-09-15T01:33:17.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082294+0000", 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.135e-05 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082550+0000", 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025634600000000002 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082593+0000", 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2518999999999997e-05 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.083323+0000", 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073059700000000004 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.083347+0000", 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3561e-05 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084300+0000", 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00095280099999999997 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 71/31 ECSubWrite(tid=252, reqid=client.4477.0:699, at_version=71'73, trim_to=0'0, roll_forward_to=71'71))", 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.312764+0000", 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7376216920000003, 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0067010990000000003, 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.312764+0000", 2024-09-15T01:33:17.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.312764+0000", 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.312766+0000", 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6649999999999999e-06 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.314419+0000", 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016526449999999999 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.314466+0000", 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6901999999999999e-05 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.314476+0000", 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0811e-05 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.314538+0000", 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1358000000000004e-05 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.314550+0000", 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2405000000000001e-05 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.315054+0000", 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00050381899999999997 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.315091+0000", 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6953999999999999e-05 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.315490+0000", 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039952100000000001 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.315515+0000", 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4343999999999999e-05 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319465+0000", 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039506749999999998 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 71/31 ECSubWrite(tid=253, reqid=client.4477.0:699, at_version=0'0, trim_to=0'0, roll_forward_to=71'73))", 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.316660+0000", 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.733726415, 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003192664, 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316660+0000", 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316660+0000", 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316660+0000", 2024-09-15T01:33:17.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5199999999999995e-07 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316687+0000", 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6944000000000002e-05 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316731+0000", 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3507999999999997e-05 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316739+0000", 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3650000000000004e-06 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316819+0000", 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9528000000000002e-05 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316833+0000", 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4319e-05 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316982+0000", 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000148567 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.317006+0000", 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4604999999999999e-05 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.317197+0000", 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019090599999999999 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.317215+0000", 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7759000000000001e-05 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319852+0000", 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026373109999999998 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 71/69 ECSubWrite(tid=179, reqid=client.4477.0:706, at_version=71'25, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:17.153 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.319734+0000", 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7306523839999999, 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035915090000000001, 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319734+0000", 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319734+0000", 2024-09-15T01:33:17.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319735+0000", 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1939999999999999e-06 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319844+0000", 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109064 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319932+0000", 2024-09-15T01:33:17.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8086999999999999e-05 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319940+0000", 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2330000000000006e-06 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.320028+0000", 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7837000000000006e-05 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.320042+0000", 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4089e-05 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.320346+0000", 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030355600000000002 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.320376+0000", 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0431999999999999e-05 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.320627+0000", 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025115000000000001 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321467+0000", 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083993499999999999 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.323325+0000", 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018579320000000001 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.158 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 71/69 ECSubWrite(tid=73, reqid=client.4477.0:701, at_version=71'45, trim_to=0'0, roll_forward_to=70'43))", 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.321927+0000", 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.728458496, 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0074903310000000002, 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321927+0000", 2024-09-15T01:33:17.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321927+0000", 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321930+0000", 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1449999999999998e-06 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322708+0000", 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077836300000000001 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322764+0000", 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6158e-05 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322784+0000", 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9809e-05 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322869+0000", 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4559000000000001e-05 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322880+0000", 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1922e-05 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.323756+0000", 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087522499999999998 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.323799+0000", 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3216000000000001e-05 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.324311+0000", 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000512132 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.324366+0000", 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5204999999999997e-05 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:17.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:17.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:17.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.329418+0000", 2024-09-15T01:33:17.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050515969999999997 2024-09-15T01:33:17.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:17.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:17.164 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:17.165 INFO:tasks.rados.rados.0.smithi175.stdout:286: done (1 left) 2024-09-15T01:33:17.165 INFO:tasks.rados.rados.0.smithi175.stdout:287: done (0 left) 2024-09-15T01:33:17.165 INFO:tasks.rados.rados.0.smithi175.stdout:288: snap_remove snap 16 2024-09-15T01:33:17.166 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:33:17.167 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:17.164+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:17.266 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:17.264+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:17.277 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:33:17.277 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:17.285 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:33:17.369 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:17.367+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:17.422 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:17.423 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:17.423 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:33:17.423 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:33:17.423 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:17.431 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:33:17.468 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:17.467+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:17.568 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:17.568 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:33:17.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:33:17.568 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:33:17.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.568 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as2 44/38 ECSubWrite(tid=24, reqid=osd.3.0:24, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:17.568 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.240642+0000", 2024-09-15T01:33:17.568 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.324579811, 2024-09-15T01:33:17.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018810250000000001, 2024-09-15T01:33:17.569 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.569 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.569 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240642+0000", 2024-09-15T01:33:17.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240642+0000", 2024-09-15T01:33:17.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240642+0000", 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0599999999999999e-07 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240655+0000", 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2584e-05 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240684+0000", 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9045999999999999e-05 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240688+0000", 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9929999999999997e-06 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240721+0000", 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2867999999999998e-05 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.240729+0000", 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6489999999999997e-06 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242915+0000", 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021854550000000002 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243256+0000", 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034129400000000002 2024-09-15T01:33:17.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243353+0000", 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6564999999999994e-05 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243364+0000", 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1262e-05 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259452+0000", 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016088327999999999 2024-09-15T01:33:17.573 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as2 44/38 ECSubWrite(tid=25, reqid=osd.3.0:25, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.242005+0000", 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.323217346, 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022660409999999999, 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242005+0000", 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242005+0000", 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242005+0000", 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242324+0000", 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031827 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242413+0000", 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9316999999999999e-05 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242420+0000", 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.435e-06 2024-09-15T01:33:17.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242951+0000", 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00053097300000000001 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242959+0000", 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8539999999999997e-06 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243124+0000", 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016528699999999999 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243257+0000", 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013250299999999999 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243353+0000", 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6482000000000002e-05 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243380+0000", 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6288e-05 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264665+0000", 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021285314 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs2 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:17.579 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.242999+0000", 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.322222902, 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022065234, 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242999+0000", 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242999+0000", 2024-09-15T01:33:17.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.242999+0000", 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2999999999999999e-07 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243014+0000", 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5074e-05 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243068+0000", 2024-09-15T01:33:17.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3211999999999998e-05 2024-09-15T01:33:17.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243076+0000", 2024-09-15T01:33:17.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5920000000000001e-06 2024-09-15T01:33:17.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243113+0000", 2024-09-15T01:33:17.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6418000000000003e-05 2024-09-15T01:33:17.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243120+0000", 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8380000000000002e-06 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243259+0000", 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013872 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243356+0000", 2024-09-15T01:33:17.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6434999999999994e-05 2024-09-15T01:33:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244107+0000", 2024-09-15T01:33:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075110800000000001 2024-09-15T01:33:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244123+0000", 2024-09-15T01:33:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6833999999999998e-05 2024-09-15T01:33:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.265064+0000", 2024-09-15T01:33:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020940772999999999 2024-09-15T01:33:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 44/31 ECSubWrite(tid=30, reqid=osd.5.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.243886+0000", 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.321336380999998, 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020918115000000001, 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243886+0000", 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243886+0000", 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243886+0000", 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3e-07 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243897+0000", 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1097999999999999e-05 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243928+0000", 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0669000000000001e-05 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243934+0000", 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9630000000000003e-06 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243975+0000", 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0602000000000002e-05 2024-09-15T01:33:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243983+0000", 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2849999999999995e-06 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244045+0000", 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2473999999999999e-05 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244109+0000", 2024-09-15T01:33:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3540000000000005e-05 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244144+0000", 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4715999999999998e-05 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244415+0000", 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027119900000000002 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264804+0000", 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020388838999999999 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 44/31 ECSubWrite(tid=63, reqid=osd.14.0:62, at_version=0'0, trim_to=0'0, roll_forward_to=44'24))", 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.243950+0000", 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.321271753000001, 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021221151000000001, 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243950+0000", 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243950+0000", 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243951+0000", 2024-09-15T01:33:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243958+0000", 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6890000000000004e-06 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243980+0000", 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1979000000000001e-05 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.243985+0000", 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1050000000000003e-06 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244044+0000", 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9011000000000001e-05 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244053+0000", 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7199999999999995e-06 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244163+0000", 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011003800000000001 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244178+0000", 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4806e-05 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244202+0000", 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4312e-05 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.244347+0000", 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000144532 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.265171+0000", 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020824529000000001 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.9s1 44/31 ECSubWrite(tid=64, reqid=osd.14.0:60, at_version=0'0, trim_to=0'0, roll_forward_to=44'18))", 2024-09-15T01:33:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.245084+0000", 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.320137667000001, 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020157773, 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245084+0000", 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245084+0000", 2024-09-15T01:33:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245085+0000", 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6400000000000002e-07 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245094+0000", 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1519999999999998e-06 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245123+0000", 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9331000000000001e-05 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245130+0000", 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2210000000000002e-06 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245176+0000", 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6777999999999998e-05 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245185+0000", 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0250000000000008e-06 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245277+0000", 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1860000000000005e-05 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245286+0000", 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5720000000000003e-06 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245316+0000", 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0382000000000002e-05 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.245328+0000", 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1381e-05 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.265242+0000", 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019914406999999999 2024-09-15T01:33:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs2 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.247258+0000", 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.317963482, 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018265526000000001, 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247258+0000", 2024-09-15T01:33:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247258+0000", 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247259+0000", 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247269+0000", 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4029999999999996e-06 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247297+0000", 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8841000000000001e-05 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247304+0000", 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819999999999998e-06 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247349+0000", 2024-09-15T01:33:17.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5028999999999999e-05 2024-09-15T01:33:17.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247360+0000", 2024-09-15T01:33:17.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0708e-05 2024-09-15T01:33:17.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247486+0000", 2024-09-15T01:33:17.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000126434 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247491+0000", 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2650000000000004e-06 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247518+0000", 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6903999999999999e-05 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247527+0000", 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4419999999999998e-06 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.265524+0000", 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017997168000000001 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as2 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.247665+0000", 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.317556912000001, 2024-09-15T01:33:17.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018084280000000001, 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247665+0000", 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247665+0000", 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247666+0000", 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8599999999999998e-07 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247673+0000", 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7940000000000003e-06 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247695+0000", 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1219000000000001e-05 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247702+0000", 2024-09-15T01:33:17.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3270000000000003e-06 2024-09-15T01:33:17.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247740+0000", 2024-09-15T01:33:17.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8415999999999999e-05 2024-09-15T01:33:17.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247750+0000", 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247845+0000", 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4376000000000006e-05 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247849+0000", 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.177e-06 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247864+0000", 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5688999999999998e-05 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247871+0000", 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3430000000000001e-06 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.265749+0000", 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017878508000000001 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s2 44/38 ECSubWrite(tid=21, reqid=osd.0.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.248814+0000", 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.316408234000001, 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017093291, 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248814+0000", 2024-09-15T01:33:17.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248814+0000", 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248814+0000", 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.29e-07 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248819+0000", 2024-09-15T01:33:17.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0329999999999999e-06 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248875+0000", 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6626000000000003e-05 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248881+0000", 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5369999999999997e-06 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248944+0000", 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2988000000000001e-05 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.248950+0000", 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5450000000000003e-06 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249016+0000", 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6206999999999998e-05 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250716+0000", 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017000069999999999 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250782+0000", 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5853999999999997e-05 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250989+0000", 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020694499999999999 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.265907+0000", 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01491842 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 44/31 ECSubWrite(tid=65, reqid=osd.14.0:65, at_version=44'26, trim_to=0'0, roll_forward_to=44'24))", 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.249459+0000", 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.315763009000001, 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016594660000000001, 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.618 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249459+0000", 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249459+0000", 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249459+0000", 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.75e-07 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249485+0000", 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5675999999999999e-05 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249506+0000", 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.126e-05 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249510+0000", 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7299999999999999e-06 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249538+0000", 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7470999999999998e-05 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.249546+0000", 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6489999999999997e-06 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250969+0000", 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014230110000000001 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251007+0000", 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7552999999999998e-05 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251122+0000", 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114968 2024-09-15T01:33:17.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251141+0000", 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8926000000000001e-05 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.266054+0000", 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014912941000000001 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 44/31 ECSubWrite(tid=28, reqid=osd.8.0:28, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.250028+0000", 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.315193437000001, 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023067219, 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250028+0000", 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250028+0000", 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250029+0000", 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.67e-07 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250046+0000", 2024-09-15T01:33:17.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7005000000000001e-05 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250068+0000", 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2430999999999999e-05 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250072+0000", 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4149999999999999e-06 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250119+0000", 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7848e-05 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250130+0000", 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0742e-05 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250986+0000", 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085551500000000005 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251007+0000", 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1841999999999999e-05 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251122+0000", 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114836 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251166+0000", 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3868000000000001e-05 2024-09-15T01:33:17.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.273096+0000", 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021929549999999999 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.251582+0000", 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.313639740999999, 2024-09-15T01:33:17.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024275702, 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251582+0000", 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251582+0000", 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251583+0000", 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5400000000000001e-07 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251611+0000", 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8152000000000001e-05 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251641+0000", 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9611000000000002e-05 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251650+0000", 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3170000000000008e-06 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251681+0000", 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1449000000000003e-05 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251687+0000", 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1140000000000002e-06 2024-09-15T01:33:17.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.254374+0000", 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026860320000000001 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.254381+0000", 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.413e-06 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256547+0000", 2024-09-15T01:33:17.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021657069999999998 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256579+0000", 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2326999999999997e-05 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275858+0000", 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019278925999999998 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 44/31 ECSubWrite(tid=30, reqid=osd.8.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.257425+0000", 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.307797356999998, 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018668435000000001, 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257425+0000", 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257425+0000", 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257425+0000", 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257435+0000", 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3149999999999998e-06 2024-09-15T01:33:17.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257458+0000", 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3363e-05 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257462+0000", 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9960000000000004e-06 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257503+0000", 2024-09-15T01:33:17.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1474000000000003e-05 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257508+0000", 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9069999999999998e-06 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257573+0000", 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4609000000000004e-05 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257578+0000", 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5550000000000002e-06 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259196+0000", 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001617159 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259225+0000", 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9567999999999999e-05 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.276093+0000", 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016867785999999999 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.270894+0000", 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.294327689999999, 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016536773000000001, 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.640 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270894+0000", 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270894+0000", 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270895+0000", 2024-09-15T01:33:17.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4000000000000001e-07 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270911+0000", 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6050999999999999e-05 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270944+0000", 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2478999999999999e-05 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270950+0000", 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.669e-06 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270992+0000", 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1894999999999998e-05 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271010+0000", 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7623000000000001e-05 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271095+0000", 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5515999999999999e-05 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272837+0000", 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017415589999999999 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275485+0000", 2024-09-15T01:33:17.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026481550000000001 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275503+0000", 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8428e-05 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.287431+0000", 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011927658000000001 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s2 44/38 ECSubWrite(tid=22, reqid=client.4477.0:293, at_version=44'14, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.271033+0000", 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.294189132, 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015139394, 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271033+0000", 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271033+0000", 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271033+0000", 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6000000000000003e-07 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271236+0000", 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-15T01:33:17.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271269+0000", 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2851000000000001e-05 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271276+0000", 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2389999999999996e-06 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271317+0000", 2024-09-15T01:33:17.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1062000000000001e-05 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271326+0000", 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8789999999999992e-06 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271531+0000", 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000204377 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272837+0000", 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001306478 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275485+0000", 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026479120000000001 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275527+0000", 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2113000000000002e-05 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.286172+0000", 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010644941999999999 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 44/31 ECSubWrite(tid=34, reqid=client.4477.0:296, at_version=44'25, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.271419+0000", 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.293802958000001, 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016338892000000001, 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.651 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271419+0000", 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271419+0000", 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271420+0000", 2024-09-15T01:33:17.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.68e-07 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271435+0000", 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5e-05 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271468+0000", 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3432000000000001e-05 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271473+0000", 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4480000000000002e-06 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271528+0000", 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4490999999999997e-05 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271536+0000", 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2640000000000008e-06 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271620+0000", 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3562999999999999e-05 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272838+0000", 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012179389999999999 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275487+0000", 2024-09-15T01:33:17.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026495999999999998 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275545+0000", 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7221e-05 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.287758+0000", 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012213366 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.272352+0000", 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.292869570000001, 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015840379000000002, 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272352+0000", 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272352+0000", 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272353+0000", 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272365+0000", 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2207e-05 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272387+0000", 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2039e-05 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272393+0000", 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5419999999999996e-06 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272432+0000", 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8897999999999999e-05 2024-09-15T01:33:17.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272442+0000", 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0052e-05 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272515+0000", 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3467000000000007e-05 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272838+0000", 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032300900000000002 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275489+0000", 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026506490000000001 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275530+0000", 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1037e-05 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.288193+0000", 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012662995 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s3 47/31 ECSubWrite(tid=51, reqid=client.4477.0:371, at_version=47'39, trim_to=0'0, roll_forward_to=47'36))", 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:49.359666+0000", 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.205555947000001, 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.65040829099999997, 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.359666+0000", 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.359666+0000", 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.359667+0000", 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.92e-07 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.359695+0000", 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7982000000000001e-05 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009537+0000", 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.64984255099999999 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009542+0000", 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999998e-06 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009587+0000", 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5257999999999999e-05 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009594+0000", 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9859999999999997e-06 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009679+0000", 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4478e-05 2024-09-15T01:33:17.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009685+0000", 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9349999999999999e-06 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009721+0000", 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6588000000000003e-05 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.009742+0000", 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0766000000000001e-05 2024-09-15T01:33:17.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.010074+0000", 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033208499999999998 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s2 47/38 ECSubWrite(tid=32, reqid=client.4477.0:386, at_version=47'21, trim_to=0'0, roll_forward_to=45'16))", 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:49.387720+0000", 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.177501846999998, 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.89050907300000004, 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.387720+0000", 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.387720+0000", 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.387721+0000", 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.387737+0000", 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6359999999999999e-05 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277497+0000", 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.88975975900000004 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277502+0000", 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6010000000000002e-06 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277615+0000", 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112551 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277628+0000", 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2486e-05 2024-09-15T01:33:17.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277734+0000", 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106314 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277769+0000", 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5200999999999997e-05 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277815+0000", 2024-09-15T01:33:17.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5788999999999998e-05 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.277835+0000", 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.014e-05 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.278229+0000", 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039418500000000002 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.4s3 69/31 ECSubRead(tid=58, to_read={3:269bd1ac:::smithi17529310-23:head=0,573440,0}, subchunks={3:269bd1ac:::smithi17529310-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:11.517498+0000", 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0477237239999999, 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.97430998899999999, 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.517498+0000", 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.517498+0000", 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.517499+0000", 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.517506+0000", 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5379999999999997e-06 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.491494+0000", 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.97398747600000002 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.491500+0000", 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.491596+0000", 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5797000000000005e-05 2024-09-15T01:33:17.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.491808+0000", 2024-09-15T01:33:17.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021268099999999999 2024-09-15T01:33:17.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.676 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.676 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.676 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:17.678 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:33:17.678 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:17.675+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:17.775 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:33:17.776 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:17.778 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:17.777+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:17.784 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:33:17.879 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:17.877+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:17.879 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:17.879 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:17.879 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:33:17.879 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:33:17.879 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:17.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:33:17.980 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:17.978+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:17.986 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:17.986 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:33:17.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:33:17.986 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:33:17.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:208 3.8s0 3:1226c857:::smithi17529310-32:head [setxattr _header (56) in=7b] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.219870+0000", 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.763331985999997, 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93580432499999999, 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 208 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219870+0000", 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219870+0000", 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219871+0000", 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9100000000000002e-07 2024-09-15T01:33:17.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219875+0000", 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7340000000000002e-06 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219882+0000", 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1729999999999998e-06 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219886+0000", 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5129999999999998e-06 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220853+0000", 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096616900000000001 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.220951+0000", 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8140000000000006e-05 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221294+0000", 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034291600000000002 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221386+0000", 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2517999999999997e-05 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.221394+0000", 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8299999999999996e-06 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.223718+0000", 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023245449999999999 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.223730+0000", 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1659e-05 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.155614+0000", 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93188401099999996 2024-09-15T01:33:17.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.155674+0000", 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0226000000000002e-05 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:209 3.8s0 3:1226c857:::smithi17529310-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:40.219933+0000", 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.763268740000001, 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.937063969, 2024-09-15T01:33:17.993 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 209 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219933+0000", 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219933+0000", 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219934+0000", 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999999e-07 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219937+0000", 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3230000000000002e-06 2024-09-15T01:33:17.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219949+0000", 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1205e-05 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.219956+0000", 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1829999999999997e-06 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.222023+0000", 2024-09-15T01:33:17.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020672009999999998 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:40.222072+0000", 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8752999999999999e-05 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.155960+0000", 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93388811299999996 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.156034+0000", 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4264999999999995e-05 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:41.156997+0000", 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096320599999999998 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:295 3.8s0 3:1226c857:::smithi17529310-32:head [rollback 8] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:17.998 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.269615+0000", 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.71358734, 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0098095839999999997, 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 295 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:17.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269615+0000", 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269615+0000", 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269615+0000", 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269618+0000", 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1599999999999998e-06 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269630+0000", 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1532000000000001e-05 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269634+0000", 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1529999999999999e-06 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270497+0000", 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000863162 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270551+0000", 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4046999999999999e-05 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270862+0000", 2024-09-15T01:33:18.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031048399999999998 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270964+0000", 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010242 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.270976+0000", 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271008+0000", 2024-09-15T01:33:18.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2227999999999997e-05 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271543+0000", 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00053470700000000004 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.277623+0000", 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060804329999999997 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.279424+0000", 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018007920000000001 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:296 3.8s0 3:1226c857:::smithi17529310-32:head [append 0~0] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.269639+0000", 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.713562970000002, 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011983048, 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 296 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269639+0000", 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269639+0000", 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269639+0000", 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269641+0000", 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0810000000000001e-06 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269647+0000", 2024-09-15T01:33:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8080000000000001e-06 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.269652+0000", 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3470000000000001e-06 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271053+0000", 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014012899999999999 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271118+0000", 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5505000000000004e-05 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271424+0000", 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030509800000000001 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271507+0000", 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3138000000000003e-05 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271523+0000", 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6511000000000002e-05 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271615+0000", 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1625000000000005e-05 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.271647+0000", 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2367999999999999e-05 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.277790+0000", 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006142659 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.281622+0000", 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038322790000000001 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s5 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.272282+0000", 2024-09-15T01:33:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.710919885999999, 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0098323179999999996, 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272282+0000", 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272282+0000", 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272283+0000", 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4099999999999998e-07 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272299+0000", 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6659000000000001e-05 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272328+0000", 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8515000000000001e-05 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272334+0000", 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272369+0000", 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5559999999999998e-05 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272379+0000", 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6809999999999993e-06 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272468+0000", 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8875999999999994e-05 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272473+0000", 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1200000000000001e-06 2024-09-15T01:33:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272511+0000", 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8408999999999998e-05 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.272521+0000", 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6309999999999998e-06 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.282114+0000", 2024-09-15T01:33:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0095934420000000006 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=51, last_complete=47'39, committed=1, applied=1))", 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:49.360072+0000", 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.623130524, 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.83144309199999999, 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.360072+0000", 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.360072+0000", 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.360072+0000", 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7199999999999998e-07 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.360075+0000", 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7700000000000002e-06 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.190654+0000", 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.83057901599999995 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.190662+0000", 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5650000000000004e-06 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.190823+0000", 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016112799999999999 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.191515+0000", 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00069194099999999995 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=52, last_complete=47'39, committed=1, applied=1))", 2024-09-15T01:33:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:50.191812+0000", 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.791389832, 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047803646999999998, 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.191812+0000", 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.191812+0000", 2024-09-15T01:33:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.191813+0000", 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8200000000000001e-07 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.191816+0000", 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0230000000000001e-06 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.239494+0000", 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047677905999999999 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.239499+0000", 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1619999999999999e-06 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.239592+0000", 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3135999999999997e-05 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.239616+0000", 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4037999999999999e-05 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:459 3.8s0 3:1c3041d9:::smithi17529310-12:head [setxattr _header (57) in=7b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:56.090711+0000", 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.892490659, 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.83095945999999998, 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 459 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090711+0000", 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090711+0000", 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090712+0000", 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3799999999999997e-07 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090715+0000", 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.19e-06 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090731+0000", 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6305999999999999e-05 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090739+0000", 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5580000000000003e-06 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.093526+0000", 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027874839999999998 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.093626+0000", 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9702000000000002e-05 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094042+0000", 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041598299999999998 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094169+0000", 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012725799999999999 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094178+0000", 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0100000000000001e-06 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094213+0000", 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4171000000000002e-05 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094316+0000", 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103417 2024-09-15T01:33:18.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.921612+0000", 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.82729596100000002 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.921671+0000", 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8882000000000003e-05 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.030 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:460 3.8s0 3:1c3041d9:::smithi17529310-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:56.090744+0000", 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.892458319999999, 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.83214597000000001, 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 460 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090744+0000", 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090744+0000", 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090744+0000", 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3499999999999996e-07 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090747+0000", 2024-09-15T01:33:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8930000000000001e-06 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090755+0000", 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0420000000000003e-06 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.090760+0000", 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7779999999999998e-06 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094227+0000", 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003466883 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.094296+0000", 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9388999999999996e-05 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.921916+0000", 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.82761973799999999 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.922012+0000", 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5470000000000006e-05 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:56.922890+0000", 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00087814200000000005 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:503 3.8s0 3:1abde0c6:::smithi17529310-42:head [copy-from ver 41] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:18.035 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.073001+0000", 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.910201395000001, 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032844862000000002, 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 503 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073001+0000", 2024-09-15T01:33:18.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073001+0000", 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073003+0000", 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9690000000000001e-06 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073009+0000", 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1299999999999998e-06 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073023+0000", 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4684999999999999e-05 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073033+0000", 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2750000000000001e-06 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073097+0000", 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4503000000000002e-05 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073188+0000", 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0748000000000006e-05 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.095640+0000", 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022451861 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096520+0000", 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00088040899999999997 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096558+0000", 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7358999999999999e-05 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096970+0000", 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041243600000000002 2024-09-15T01:33:18.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096989+0000", 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9110000000000002e-05 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.105798+0000", 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0088090970000000001 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.105846+0000", 2024-09-15T01:33:18.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7280000000000001e-05 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:504 3.8s0 3:1abde0c6:::smithi17529310-42:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.073045+0000", 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.910156913000002, 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023923209000000001, 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.042 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 504 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073045+0000", 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073045+0000", 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073046+0000", 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0100000000000005e-07 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073049+0000", 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2619999999999999e-06 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073065+0000", 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5794000000000002e-05 2024-09-15T01:33:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073070+0000", 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4729999999999999e-06 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073328+0000", 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025764599999999999 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073406+0000", 2024-09-15T01:33:18.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7825999999999997e-05 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096769+0000", 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023363063 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096883+0000", 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114212 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.096968+0000", 2024-09-15T01:33:18.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5332000000000002e-05 2024-09-15T01:33:18.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.047 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.047 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs6 55/31 ECSubWrite(tid=160, reqid=client.4477.0:512, at_version=55'53, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:18.047 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.075658+0000", 2024-09-15T01:33:18.047 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.907543649000001, 2024-09-15T01:33:18.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0092545209999999999, 2024-09-15T01:33:18.047 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.047 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075658+0000", 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075658+0000", 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075659+0000", 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6199999999999997e-07 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075674+0000", 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4739000000000001e-05 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075705+0000", 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0618999999999999e-05 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075711+0000", 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2179999999999996e-06 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075753+0000", 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2178999999999997e-05 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075763+0000", 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7839999999999998e-06 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075863+0000", 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100163 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075874+0000", 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-05 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075913+0000", 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9539000000000002e-05 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.075929+0000", 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5554999999999999e-05 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.084913+0000", 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089839229999999996 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:550 3.8s0 3:1abde0c6:::smithi17529310-42:head [rollback 19] snapc 19=[19,15] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-15T01:33:18.052 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:01.086634+0000", 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.896567587, 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0096404579999999993, 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 550 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086634+0000", 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086634+0000", 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086635+0000", 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7000000000000005e-07 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086638+0000", 2024-09-15T01:33:18.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3380000000000001e-06 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086646+0000", 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.763e-06 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086651+0000", 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.087875+0000", 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001223745 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.087950+0000", 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5459999999999999e-05 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.088528+0000", 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0005775 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.088644+0000", 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116746 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.088660+0000", 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5642999999999999e-05 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.088711+0000", 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1127e-05 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.089434+0000", 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00072320700000000002 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.090185+0000", 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075082399999999998 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.096275+0000", 2024-09-15T01:33:18.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060898439999999996 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:551 3.8s0 3:1abde0c6:::smithi17529310-42:head [append 0~0] snapc 19=[19,15] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:01.086656+0000", 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.896546221000001, 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0097169549999999993, 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.059 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 551 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086656+0000", 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086656+0000", 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086656+0000", 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086659+0000", 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.339e-06 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086665+0000", 2024-09-15T01:33:18.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0630000000000002e-06 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.086669+0000", 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9469999999999996e-06 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.088717+0000", 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020487539999999999 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.088808+0000", 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0724999999999996e-05 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.089265+0000", 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045738199999999999 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.089355+0000", 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9184999999999994e-05 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.089367+0000", 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2371999999999999e-05 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.089636+0000", 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000269225 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.089844+0000", 2024-09-15T01:33:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020802099999999999 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.090364+0000", 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051979399999999996 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:01.096373+0000", 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060087969999999997 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.3s5 66/31 ECSubWrite(tid=241, reqid=osd.13.0:241, at_version=66'51, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:08.081466+0000", 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.9017361780000002, 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0094524540000000008, 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081466+0000", 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081466+0000", 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081466+0000", 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6199999999999999e-07 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081495+0000", 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8538999999999999e-05 2024-09-15T01:33:18.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081542+0000", 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7707000000000002e-05 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081551+0000", 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2490000000000001e-06 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081607+0000", 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5988e-05 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.081617+0000", 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.023e-05 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083363+0000", 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017467559999999999 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083376+0000", 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2729e-05 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083398+0000", 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2192999999999999e-05 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083614+0000", 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021578999999999999 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.090918+0000", 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073041110000000003 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.3s5 66/31 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=66'53, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:08.082267+0000", 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.9009352990000004, 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0094909190000000004, 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.082267+0000", 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.082267+0000", 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.082267+0000", 2024-09-15T01:33:18.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.082287+0000", 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9110000000000002e-05 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.082319+0000", 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2620000000000003e-05 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.082324+0000", 2024-09-15T01:33:18.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083400+0000", 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001076013 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083407+0000", 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2199999999999997e-06 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083596+0000", 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018967699999999999 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083604+0000", 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4139999999999997e-06 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083644+0000", 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9824999999999999e-05 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.083656+0000", 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2488e-05 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:08.091758+0000", 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0081016390000000008 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.076 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:643 3.8s0 3:1c3041d9:::smithi17529310-12:head [create,setxattr XXWdXV_kiPaajZX\\PgmqNaZqSVmab^hrtRYeagZr (1894) in=40b,setxattr NdZr`e\\^kXiksh]\\XocenQrhYc]c`[proOfo`Pd^ (33) in=40b,setxattr _`NqSdldlQbUNl^ZNRba\\naZhhdraiOgpafrZb[o (103) in=40b,setxattr iadimrq_[jOPUblhnVqTYqQNedg (1647) in=27b,setxattr `e\\rtQi]nmoP\\cm]oRYN`[]^qVUTsU^_oXm]h\\ec (6452) in=40b,setxattr ppZXc_dYi_WWXcUdeqcpck_ceUVa^XftgnfgdP[q (3576) in=40b,setxattr hehZsP\\ajqhe^nfrZjQmsnfWoqq\\e\\NV^rOaZYtj (1787) in=40b,setxattr a]Smbg`^k`emWabQRYcpr_lQj`eh`[r\\riqa_sfd (1015) in=40b,setxattr eqsaZkZ[`t^R^grhYOVrVsspVQQp_Zo\\loUV[\\U[ (982) in=40b,setxattr Rn`k`cSYQj`XOZdn_mSPrqQtoUpWTk[^[t (1647) in=34b,setxattr ]TtVabQgUTabfmqae[UVe`cNsqqYfdh]\\sVoeXQr (383) in=40b,setxattr iRn][N_NY\\ZhmYdQ[[ptTP_NW^Z]_roZnOpcmR_Y (19016) in=40b,setxattr T_b_WhnZnkaaYkNmmOUnaZa`dhRhQSshYQVQnVql (13472) in=40b,setxattr \\hZXh`dT^W[c\\XodZZ_d\\QO\\\\[PVWthO_WVOPSQd (1681) in=40b,setxattr ctSOqNY_dUqTOjgXYjqgslgNRRPoT\\\\OXPPWrtt_ (522) in=40b,setxattr f\\hTOf^sbRaYeUntSV^WhUo]qV`bSNermptfXs]Q (1645) in=40b,setxattr ohNTdfibmYs`eUTiQa^bcYW\\aarhT]YQ_haagqpf (1119) in=40b,setxattr qqUtZZ]ataUj_[pnh\\jVtU_dRpi^g[VPNeS`Y\\hk (699) in=40b,setxattr qsgedRfqdW]]cYbWOUgh^ZleaSQiteejr^afXbUj (525) in=40b,setxattr [XmPqpoSg`fWjdlmQ_qabof]_WVcPaY[]k[ardgj (594) in=40b] snapc 21=[21,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:11.517098+0000", 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.4661038020000001, 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.58675648000000002, 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 643 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.517098+0000", 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.517098+0000", 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.517100+0000", 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2890000000000001e-06 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.517235+0000", 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013587300000000001 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.517248+0000", 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2781e-05 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.517254+0000", 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.049e-06 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.517294+0000", 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0003000000000003e-05 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.517374+0000", 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9895999999999995e-05 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.518105+0000", 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00073047100000000001 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.518301+0000", 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019682400000000001 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.518314+0000", 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2849e-05 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.518457+0000", 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014252000000000001 2024-09-15T01:33:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.518471+0000", 2024-09-15T01:33:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4075e-05 2024-09-15T01:33:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103765+0000", 2024-09-15T01:33:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.58529396300000003 2024-09-15T01:33:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103855+0000", 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9887000000000002e-05 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 69/31 ECSubWriteReply(tid=87, last_complete=69'67, committed=1, applied=1))", 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:11.519517+0000", 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.4636852840000003, 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.58472048300000001, 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.519517+0000", 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.519517+0000", 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.519517+0000", 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3900000000000005e-07 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:11.519522+0000", 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2590000000000002e-06 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103508+0000", 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.58398663200000001 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103518+0000", 2024-09-15T01:33:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0025e-05 2024-09-15T01:33:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103600+0000", 2024-09-15T01:33:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1827000000000001e-05 2024-09-15T01:33:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104237+0000", 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000637201 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:680 3.8s0 3:1226c857:::smithi17529310-32:head [rollback 1e] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.273083+0000", 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.7101188910000005, 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013870413, 2024-09-15T01:33:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 680 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.273083+0000", 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.273083+0000", 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.273084+0000", 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000002e-07 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.273086+0000", 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7420000000000002e-06 2024-09-15T01:33:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.273093+0000", 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4330000000000001e-06 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.273097+0000", 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9670000000000003e-06 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.274066+0000", 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096898199999999996 2024-09-15T01:33:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.274162+0000", 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6168000000000005e-05 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.274747+0000", 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00058471100000000002 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.274861+0000", 2024-09-15T01:33:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011485 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.274868+0000", 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.348e-06 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.274901+0000", 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3343000000000002e-05 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.275389+0000", 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00048741499999999999 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.276687+0000", 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012983999999999999 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.286954+0000", 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010266588 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:681 3.8s0 3:1226c857:::smithi17529310-32:head [append 0~0] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.273110+0000", 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.7100924509999995, 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014057424000000001, 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 681 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.273110+0000", 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.273110+0000", 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.273110+0000", 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-09-15T01:33:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.273112+0000", 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2110000000000001e-06 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.273118+0000", 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5829999999999997e-06 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.273122+0000", 2024-09-15T01:33:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8670000000000004e-06 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.274926+0000", 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001804267 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.274992+0000", 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819000000000001e-05 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.275236+0000", 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024470900000000002 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.275345+0000", 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010857999999999999 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.275351+0000", 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.246e-06 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.275375+0000", 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4019000000000002e-05 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.275389+0000", 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4054e-05 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.276813+0000", 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014233690000000001 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.287167+0000", 2024-09-15T01:33:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010354303 2024-09-15T01:33:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.100 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:18.100 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:33:18.102 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:18.099+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:18.103 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:33:18.104 INFO:tasks.rados.rados.0.smithi175.stdout:288: done (0 left) 2024-09-15T01:33:18.104 INFO:tasks.rados.rados.0.smithi175.stdout:289: rollback oid 2 current snap is 23 2024-09-15T01:33:18.104 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 2 to 20 2024-09-15T01:33:18.104 INFO:tasks.rados.rados.0.smithi175.stdout:290: rollback oid 14 current snap is 23 2024-09-15T01:33:18.104 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 14 to 12 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:291: copy_from oid 19 from oid 12 current snap is 23 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:292: rollback oid 11 current snap is 23 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 11 to 21 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:293: rollback oid 49 current snap is 23 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 49 to 21 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:294: snap_create 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:292: finishing rollback tid 1 to smithi17529310-11 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 11 v 18 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:289: finishing rollback tid 0 to smithi17529310-2 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:289: finishing rollback tid 1 to smithi17529310-2 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:289: finishing rollback tid 2 to smithi17529310-2 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 2 v 151 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-15T01:33:18.105 INFO:tasks.rados.rados.0.smithi175.stdout:293: finishing rollback tid 0 to smithi17529310-49 2024-09-15T01:33:18.106 INFO:tasks.rados.rados.0.smithi175.stdout:293: finishing rollback tid 1 to smithi17529310-49 2024-09-15T01:33:18.106 INFO:tasks.rados.rados.0.smithi175.stdout:293: finishing rollback tid 2 to smithi17529310-49 2024-09-15T01:33:18.106 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 49 v 155 (ObjNum 101 snap 10 seq_num 101) dirty exists 2024-09-15T01:33:18.107 INFO:tasks.rados.rados.0.smithi175.stdout:291: finishing copy_from racing read to smithi17529310-19 2024-09-15T01:33:18.108 INFO:tasks.rados.rados.0.smithi175.stdout:291: finishing copy_from to smithi17529310-19 2024-09-15T01:33:18.108 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 19 v 67 (ObjNum 98 snap 10 seq_num 98) dirty exists 2024-09-15T01:33:18.202 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:18.200+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:18.203 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:18.203 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:18.203 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:33:18.203 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:18.212 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:33:18.302 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:18.301+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:18.309 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:18.309 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:18.309 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:33:18.309 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:33:18.309 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:18.317 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:33:18.404 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:18.402+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:18.416 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=1, reqid=client.4477.0:28, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.234651241999998, 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018499090000000001, 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:33:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179245+0000", 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2150000000000006e-05 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180346+0000", 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001101196 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180357+0000", 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175000000000001e-05 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180408+0000", 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0590999999999997e-05 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180417+0000", 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8060000000000001e-06 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180553+0000", 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136324 2024-09-15T01:33:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180684+0000", 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000130853 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180745+0000", 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0971000000000003e-05 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180919+0000", 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017386800000000001 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181032+0000", 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113578 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 34/31 ECSubWrite(tid=2, reqid=client.4477.0:14, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:33:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.234584906999999, 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018048179999999999, 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:33:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179249+0000", 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.179259+0000", 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0104e-05 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180590+0000", 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013307779999999999 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180598+0000", 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180692+0000", 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3549999999999997e-05 2024-09-15T01:33:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180700+0000", 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6330000000000002e-06 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180793+0000", 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967000000000006e-05 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180801+0000", 2024-09-15T01:33:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4580000000000004e-06 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180836+0000", 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5732999999999998e-05 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.180858+0000", 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.126e-05 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.181054+0000", 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019604099999999999 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 70/69 ECSubWrite(tid=162, reqid=client.4477.0:623, at_version=70'53, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:12.085067+0000", 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.3287663350000001, 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002376258, 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.085067+0000", 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.085067+0000", 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.085068+0000", 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5499999999999998e-07 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.085150+0000", 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1889000000000002e-05 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.085182+0000", 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2320000000000002e-05 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.085189+0000", 2024-09-15T01:33:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6309999999999997e-06 2024-09-15T01:33:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.085241+0000", 2024-09-15T01:33:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1756999999999999e-05 2024-09-15T01:33:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.085250+0000", 2024-09-15T01:33:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7610000000000002e-06 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.085510+0000", 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025984700000000001 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.085518+0000", 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8399999999999995e-06 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.085574+0000", 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6048000000000003e-05 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.085596+0000", 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1729000000000002e-05 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.087444+0000", 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018477809999999999 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 70/69 ECSubWrite(tid=71, reqid=client.4477.0:636, at_version=70'43, trim_to=0'0, roll_forward_to=70'40))", 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:12.102080+0000", 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.3117537749999997, 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050273510000000002, 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.102080+0000", 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.102080+0000", 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.102081+0000", 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6899999999999996e-07 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103542+0000", 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014608170000000001 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103579+0000", 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7376999999999998e-05 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103587+0000", 2024-09-15T01:33:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2640000000000008e-06 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103636+0000", 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8334000000000002e-05 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103646+0000", 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0010000000000001e-05 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103944+0000", 2024-09-15T01:33:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029775800000000002 2024-09-15T01:33:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103952+0000", 2024-09-15T01:33:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7900000000000005e-06 2024-09-15T01:33:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104702+0000", 2024-09-15T01:33:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00074918900000000004 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104723+0000", 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1610000000000001e-05 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.107107+0000", 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023842329999999999 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 71/69 ECSubWrite(tid=51, reqid=client.4477.0:678, at_version=71'33, trim_to=0'0, roll_forward_to=70'31))", 2024-09-15T01:33:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.271454+0000", 2024-09-15T01:33:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.142379408, 2024-09-15T01:33:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050481479999999997, 2024-09-15T01:33:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271454+0000", 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271454+0000", 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271455+0000", 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6599999999999996e-07 2024-09-15T01:33:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271510+0000", 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4975999999999997e-05 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271546+0000", 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6066999999999999e-05 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271553+0000", 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2439999999999996e-06 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271623+0000", 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9549999999999996e-05 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271634+0000", 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1399e-05 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271824+0000", 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000189877 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271834+0000", 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5990000000000008e-06 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271871+0000", 2024-09-15T01:33:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7718e-05 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271888+0000", 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6881e-05 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.276503+0000", 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046142709999999997 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 71/69 ECSubWrite(tid=52, reqid=client.4477.0:678, at_version=0'0, trim_to=0'0, roll_forward_to=71'33))", 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.272679+0000", 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.1411548849999997, 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039511759999999998, 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.272679+0000", 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.272679+0000", 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.272680+0000", 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1900000000000002e-07 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.272689+0000", 2024-09-15T01:33:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6059999999999992e-06 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.272717+0000", 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7864e-05 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.272723+0000", 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5910000000000003e-06 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.272766+0000", 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3112999999999999e-05 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.272775+0000", 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5070000000000006e-06 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.272875+0000", 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9703000000000004e-05 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.272882+0000", 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0520000000000004e-06 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.272907+0000", 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5253e-05 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.272922+0000", 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4754e-05 2024-09-15T01:33:18.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.276630+0000", 2024-09-15T01:33:18.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037080139999999999 2024-09-15T01:33:18.451 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.451 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.451 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.451 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 71/69 ECSubWrite(tid=174, reqid=client.4477.0:696, at_version=71'57, trim_to=0'0, roll_forward_to=71'55))", 2024-09-15T01:33:18.451 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.305497+0000", 2024-09-15T01:33:18.451 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.1083369970000003, 2024-09-15T01:33:18.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002208921, 2024-09-15T01:33:18.452 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.452 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.452 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.305497+0000", 2024-09-15T01:33:18.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.305497+0000", 2024-09-15T01:33:18.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.305498+0000", 2024-09-15T01:33:18.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4399999999999999e-07 2024-09-15T01:33:18.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.305515+0000", 2024-09-15T01:33:18.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7844000000000001e-05 2024-09-15T01:33:18.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.305545+0000", 2024-09-15T01:33:18.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9556e-05 2024-09-15T01:33:18.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.305552+0000", 2024-09-15T01:33:18.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6259999999999997e-06 2024-09-15T01:33:18.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.305603+0000", 2024-09-15T01:33:18.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1765000000000003e-05 2024-09-15T01:33:18.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.305610+0000", 2024-09-15T01:33:18.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4919999999999998e-06 2024-09-15T01:33:18.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.305840+0000", 2024-09-15T01:33:18.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022968199999999999 2024-09-15T01:33:18.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.305852+0000", 2024-09-15T01:33:18.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2357999999999999e-05 2024-09-15T01:33:18.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.305896+0000", 2024-09-15T01:33:18.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3730999999999999e-05 2024-09-15T01:33:18.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.305917+0000", 2024-09-15T01:33:18.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1631e-05 2024-09-15T01:33:18.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307706+0000", 2024-09-15T01:33:18.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001788392 2024-09-15T01:33:18.458 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.458 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.458 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.458 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 71/69 ECSubWrite(tid=175, reqid=client.4477.0:697, at_version=71'58, trim_to=0'0, roll_forward_to=71'55))", 2024-09-15T01:33:18.458 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.307088+0000", 2024-09-15T01:33:18.458 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.1067453560000002, 2024-09-15T01:33:18.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049056919999999997, 2024-09-15T01:33:18.458 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.459 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.459 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307088+0000", 2024-09-15T01:33:18.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307088+0000", 2024-09-15T01:33:18.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307090+0000", 2024-09-15T01:33:18.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.099e-06 2024-09-15T01:33:18.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307121+0000", 2024-09-15T01:33:18.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1538000000000002e-05 2024-09-15T01:33:18.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307175+0000", 2024-09-15T01:33:18.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3906999999999997e-05 2024-09-15T01:33:18.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307183+0000", 2024-09-15T01:33:18.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7549999999999994e-06 2024-09-15T01:33:18.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307247+0000", 2024-09-15T01:33:18.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4070000000000002e-05 2024-09-15T01:33:18.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307257+0000", 2024-09-15T01:33:18.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9729999999999992e-06 2024-09-15T01:33:18.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307362+0000", 2024-09-15T01:33:18.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010547099999999999 2024-09-15T01:33:18.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307371+0000", 2024-09-15T01:33:18.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.439e-06 2024-09-15T01:33:18.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307402+0000", 2024-09-15T01:33:18.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1121999999999998e-05 2024-09-15T01:33:18.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307425+0000", 2024-09-15T01:33:18.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3598999999999999e-05 2024-09-15T01:33:18.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.311994+0000", 2024-09-15T01:33:18.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045687189999999997 2024-09-15T01:33:18.465 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.465 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.465 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.465 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 71/69 ECSubWrite(tid=176, reqid=client.4477.0:698, at_version=71'59, trim_to=0'0, roll_forward_to=71'55))", 2024-09-15T01:33:18.465 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.307905+0000", 2024-09-15T01:33:18.465 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.1059284829999996, 2024-09-15T01:33:18.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042070679999999996, 2024-09-15T01:33:18.465 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.465 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.465 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307905+0000", 2024-09-15T01:33:18.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307905+0000", 2024-09-15T01:33:18.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307921+0000", 2024-09-15T01:33:18.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971999999999999e-05 2024-09-15T01:33:18.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307939+0000", 2024-09-15T01:33:18.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7288000000000001e-05 2024-09-15T01:33:18.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307965+0000", 2024-09-15T01:33:18.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6679999999999999e-05 2024-09-15T01:33:18.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307971+0000", 2024-09-15T01:33:18.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8629999999999996e-06 2024-09-15T01:33:18.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.308006+0000", 2024-09-15T01:33:18.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4644999999999998e-05 2024-09-15T01:33:18.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.308015+0000", 2024-09-15T01:33:18.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4790000000000002e-06 2024-09-15T01:33:18.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.308112+0000", 2024-09-15T01:33:18.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7194000000000005e-05 2024-09-15T01:33:18.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.308119+0000", 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5350000000000001e-06 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.308151+0000", 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1572000000000002e-05 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.308165+0000", 2024-09-15T01:33:18.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4545000000000001e-05 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.312112+0000", 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003947295 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 71/69 ECSubWrite(tid=179, reqid=client.4477.0:706, at_version=71'25, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.319527+0000", 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0943067480000002, 2024-09-15T01:33:18.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042633009999999997, 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.319527+0000", 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.319527+0000", 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.319528+0000", 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6400000000000001e-07 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.320017+0000", 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00048872100000000001 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.320043+0000", 2024-09-15T01:33:18.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6448000000000002e-05 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.320048+0000", 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5289999999999999e-06 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.320103+0000", 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4797999999999999e-05 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.320116+0000", 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2588999999999999e-05 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.320426+0000", 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031022500000000002 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.320435+0000", 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6500000000000002e-06 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.320519+0000", 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4444000000000007e-05 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.320543+0000", 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3688000000000001e-05 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.323790+0000", 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032474449999999998 2024-09-15T01:33:18.476 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.477 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.477 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.477 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 71/69 ECSubWrite(tid=73, reqid=client.4477.0:701, at_version=71'45, trim_to=0'0, roll_forward_to=70'43))", 2024-09-15T01:33:18.477 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.321470+0000", 2024-09-15T01:33:18.477 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0923636810000001, 2024-09-15T01:33:18.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012701758, 2024-09-15T01:33:18.477 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.477 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.477 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.321470+0000", 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.321470+0000", 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.321470+0000", 2024-09-15T01:33:18.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3299999999999998e-07 2024-09-15T01:33:18.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.323991+0000", 2024-09-15T01:33:18.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025208689999999998 2024-09-15T01:33:18.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324019+0000", 2024-09-15T01:33:18.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7208000000000001e-05 2024-09-15T01:33:18.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324025+0000", 2024-09-15T01:33:18.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1959999999999996e-06 2024-09-15T01:33:18.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324092+0000", 2024-09-15T01:33:18.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7300999999999998e-05 2024-09-15T01:33:18.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324105+0000", 2024-09-15T01:33:18.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2617e-05 2024-09-15T01:33:18.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324615+0000", 2024-09-15T01:33:18.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051016099999999997 2024-09-15T01:33:18.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324624+0000", 2024-09-15T01:33:18.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6039999999999999e-06 2024-09-15T01:33:18.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324947+0000", 2024-09-15T01:33:18.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032275400000000002 2024-09-15T01:33:18.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324963+0000", 2024-09-15T01:33:18.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5491000000000001e-05 2024-09-15T01:33:18.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.334172+0000", 2024-09-15T01:33:18.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0092092240000000002 2024-09-15T01:33:18.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.483 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.483 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 71/69 ECSubWrite(tid=181, reqid=client.4477.0:708, at_version=71'27, trim_to=0'0, roll_forward_to=71'25))", 2024-09-15T01:33:18.484 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.326902+0000", 2024-09-15T01:33:18.484 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0869314450000003, 2024-09-15T01:33:18.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003201602, 2024-09-15T01:33:18.484 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.484 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.484 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.326902+0000", 2024-09-15T01:33:18.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.326902+0000", 2024-09-15T01:33:18.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.326907+0000", 2024-09-15T01:33:18.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4479999999999996e-06 2024-09-15T01:33:18.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.327022+0000", 2024-09-15T01:33:18.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115155 2024-09-15T01:33:18.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.327047+0000", 2024-09-15T01:33:18.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5224000000000001e-05 2024-09-15T01:33:18.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.327052+0000", 2024-09-15T01:33:18.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9219999999999996e-06 2024-09-15T01:33:18.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.327093+0000", 2024-09-15T01:33:18.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1004999999999998e-05 2024-09-15T01:33:18.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.327101+0000", 2024-09-15T01:33:18.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5209999999999997e-06 2024-09-15T01:33:18.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.327213+0000", 2024-09-15T01:33:18.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112199 2024-09-15T01:33:18.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.327219+0000", 2024-09-15T01:33:18.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4250000000000003e-06 2024-09-15T01:33:18.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.329146+0000", 2024-09-15T01:33:18.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019265319999999999 2024-09-15T01:33:18.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.329160+0000", 2024-09-15T01:33:18.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4647000000000001e-05 2024-09-15T01:33:18.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.330104+0000", 2024-09-15T01:33:18.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00094352399999999995 2024-09-15T01:33:18.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.490 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.490 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 71/69 ECSubWrite(tid=74, reqid=client.4477.0:701, at_version=0'0, trim_to=0'0, roll_forward_to=71'45))", 2024-09-15T01:33:18.490 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.330184+0000", 2024-09-15T01:33:18.490 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0836502379999997, 2024-09-15T01:33:18.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041451639999999998, 2024-09-15T01:33:18.491 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.491 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.491 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.330184+0000", 2024-09-15T01:33:18.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.330184+0000", 2024-09-15T01:33:18.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.330184+0000", 2024-09-15T01:33:18.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3399999999999999e-07 2024-09-15T01:33:18.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.330194+0000", 2024-09-15T01:33:18.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7729999999999994e-06 2024-09-15T01:33:18.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.330218+0000", 2024-09-15T01:33:18.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4212000000000001e-05 2024-09-15T01:33:18.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.330226+0000", 2024-09-15T01:33:18.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8159999999999994e-06 2024-09-15T01:33:18.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.330288+0000", 2024-09-15T01:33:18.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1809000000000004e-05 2024-09-15T01:33:18.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.330299+0000", 2024-09-15T01:33:18.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1459e-05 2024-09-15T01:33:18.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.330399+0000", 2024-09-15T01:33:18.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9751000000000001e-05 2024-09-15T01:33:18.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.330406+0000", 2024-09-15T01:33:18.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1169999999999998e-06 2024-09-15T01:33:18.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331411+0000", 2024-09-15T01:33:18.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001004703 2024-09-15T01:33:18.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331425+0000", 2024-09-15T01:33:18.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3891e-05 2024-09-15T01:33:18.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.334329+0000", 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002903999 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 71/69 ECSubWrite(tid=184, reqid=client.4477.0:710, at_version=0'0, trim_to=0'0, roll_forward_to=71'29))", 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.331203+0000", 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0826304059999998, 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030096459999999999, 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331203+0000", 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331203+0000", 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331204+0000", 2024-09-15T01:33:18.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6400000000000002e-07 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331214+0000", 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0052e-05 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331244+0000", 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9439000000000001e-05 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331249+0000", 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5890000000000002e-06 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331279+0000", 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9978e-05 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331285+0000", 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2129999999999996e-06 2024-09-15T01:33:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331353+0000", 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7558999999999995e-05 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331413+0000", 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9969000000000001e-05 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331458+0000", 2024-09-15T01:33:18.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5451999999999998e-05 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.331467+0000", 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6100000000000006e-06 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.334213+0000", 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027461209999999998 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.502 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 71/69 ECSubWrite(tid=186, reqid=client.4477.0:713, at_version=71'18, trim_to=0'0, roll_forward_to=70'16))", 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.400596+0000", 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0132378480000002, 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039785159999999996, 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.400596+0000", 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.400596+0000", 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.400597+0000", 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1900000000000002e-07 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.400615+0000", 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8569000000000001e-05 2024-09-15T01:33:18.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.400642+0000", 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7106000000000001e-05 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.400650+0000", 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6229999999999997e-06 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.400717+0000", 2024-09-15T01:33:18.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6748000000000005e-05 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.400724+0000", 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2280000000000001e-06 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.400934+0000", 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020998100000000001 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.400941+0000", 2024-09-15T01:33:18.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3080000000000002e-06 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.400977+0000", 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5957999999999997e-05 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.400992+0000", 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4470999999999999e-05 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.404574+0000", 2024-09-15T01:33:18.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035828050000000001 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 71/69 ECSubWrite(tid=187, reqid=client.4477.0:713, at_version=0'0, trim_to=0'0, roll_forward_to=71'18))", 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.401651+0000", 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0121831910000001, 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030431989999999999, 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.401651+0000", 2024-09-15T01:33:18.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.401651+0000", 2024-09-15T01:33:18.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.401651+0000", 2024-09-15T01:33:18.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8299999999999997e-07 2024-09-15T01:33:18.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.401659+0000", 2024-09-15T01:33:18.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1780000000000003e-06 2024-09-15T01:33:18.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.401683+0000", 2024-09-15T01:33:18.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3220000000000001e-05 2024-09-15T01:33:18.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.401700+0000", 2024-09-15T01:33:18.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7269999999999999e-05 2024-09-15T01:33:18.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.401741+0000", 2024-09-15T01:33:18.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1638999999999999e-05 2024-09-15T01:33:18.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.401748+0000", 2024-09-15T01:33:18.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.212e-06 2024-09-15T01:33:18.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.401813+0000", 2024-09-15T01:33:18.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5038999999999999e-05 2024-09-15T01:33:18.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.401819+0000", 2024-09-15T01:33:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.708e-06 2024-09-15T01:33:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.401848+0000", 2024-09-15T01:33:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8592999999999999e-05 2024-09-15T01:33:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.401862+0000", 2024-09-15T01:33:18.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4328000000000001e-05 2024-09-15T01:33:18.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.404694+0000", 2024-09-15T01:33:18.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028314289999999999 2024-09-15T01:33:18.514 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.514 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.514 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.515 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 76/69 ECSubWrite(tid=79, reqid=osd.0.0:79, at_version=76'50, trim_to=0'0, roll_forward_to=74'48))", 2024-09-15T01:33:18.515 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:18.081206+0000", 2024-09-15T01:33:18.515 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.33262740699999999, 2024-09-15T01:33:18.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023949800000000001, 2024-09-15T01:33:18.515 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.515 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.515 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.081206+0000", 2024-09-15T01:33:18.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.081206+0000", 2024-09-15T01:33:18.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.081207+0000", 2024-09-15T01:33:18.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.99e-07 2024-09-15T01:33:18.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.081230+0000", 2024-09-15T01:33:18.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3859999999999999e-05 2024-09-15T01:33:18.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.081265+0000", 2024-09-15T01:33:18.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4879999999999998e-05 2024-09-15T01:33:18.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.081271+0000", 2024-09-15T01:33:18.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4539999999999998e-06 2024-09-15T01:33:18.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.081306+0000", 2024-09-15T01:33:18.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4898999999999999e-05 2024-09-15T01:33:18.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.081314+0000", 2024-09-15T01:33:18.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9419999999999995e-06 2024-09-15T01:33:18.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.081535+0000", 2024-09-15T01:33:18.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022128 2024-09-15T01:33:18.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.081817+0000", 2024-09-15T01:33:18.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028241900000000001 2024-09-15T01:33:18.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.081847+0000", 2024-09-15T01:33:18.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9736000000000001e-05 2024-09-15T01:33:18.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.081857+0000", 2024-09-15T01:33:18.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9490000000000008e-06 2024-09-15T01:33:18.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.083601+0000", 2024-09-15T01:33:18.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001744362 2024-09-15T01:33:18.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.521 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.521 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s2 76/69 ECSubWrite(tid=198, reqid=osd.4.0:198, at_version=76'23, trim_to=0'0, roll_forward_to=73'20))", 2024-09-15T01:33:18.521 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:18.083216+0000", 2024-09-15T01:33:18.522 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.33061772699999997, 2024-09-15T01:33:18.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021522609999999999, 2024-09-15T01:33:18.522 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.522 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.522 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.083216+0000", 2024-09-15T01:33:18.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.083216+0000", 2024-09-15T01:33:18.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.083216+0000", 2024-09-15T01:33:18.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5499999999999999e-07 2024-09-15T01:33:18.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.083230+0000", 2024-09-15T01:33:18.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3125000000000001e-05 2024-09-15T01:33:18.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.083692+0000", 2024-09-15T01:33:18.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046295899999999998 2024-09-15T01:33:18.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.083698+0000", 2024-09-15T01:33:18.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7529999999999998e-06 2024-09-15T01:33:18.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.084262+0000", 2024-09-15T01:33:18.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00056424999999999997 2024-09-15T01:33:18.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.084275+0000", 2024-09-15T01:33:18.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2795e-05 2024-09-15T01:33:18.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.084479+0000", 2024-09-15T01:33:18.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020418699999999999 2024-09-15T01:33:18.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.084670+0000", 2024-09-15T01:33:18.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000190309 2024-09-15T01:33:18.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.084849+0000", 2024-09-15T01:33:18.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017888999999999999 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.084996+0000", 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000146902 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.085368+0000", 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037273599999999998 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 76/69 ECSubWrite(tid=204, reqid=client.4477.0:748, at_version=76'69, trim_to=0'0, roll_forward_to=76'67))", 2024-09-15T01:33:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:18.103204+0000", 2024-09-15T01:33:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.31062948000000001, 2024-09-15T01:33:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065162520000000002, 2024-09-15T01:33:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.103204+0000", 2024-09-15T01:33:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.103204+0000", 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.103205+0000", 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.61e-07 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105300+0000", 2024-09-15T01:33:18.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020949359999999999 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105325+0000", 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5287999999999999e-05 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105330+0000", 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6959999999999996e-06 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105379+0000", 2024-09-15T01:33:18.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8591000000000003e-05 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105389+0000", 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0919e-05 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105775+0000", 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038574300000000002 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105785+0000", 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0015e-05 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105984+0000", 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019886700000000001 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.106001+0000", 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6413999999999999e-05 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.109721+0000", 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037200219999999999 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 76/69 ECSubWrite(tid=205, reqid=client.4477.0:748, at_version=0'0, trim_to=0'0, roll_forward_to=76'69))", 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:18.107090+0000", 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.30674405399999999, 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027974789999999999, 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.533 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.107090+0000", 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.107090+0000", 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.107090+0000", 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6000000000000003e-07 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.107099+0000", 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7439999999999996e-06 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.107125+0000", 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5741e-05 2024-09-15T01:33:18.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.107131+0000", 2024-09-15T01:33:18.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2929999999999997e-06 2024-09-15T01:33:18.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.107185+0000", 2024-09-15T01:33:18.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3956999999999999e-05 2024-09-15T01:33:18.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.107198+0000", 2024-09-15T01:33:18.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2781e-05 2024-09-15T01:33:18.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.107292+0000", 2024-09-15T01:33:18.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4346999999999996e-05 2024-09-15T01:33:18.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.107298+0000", 2024-09-15T01:33:18.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0789999999999997e-06 2024-09-15T01:33:18.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.107353+0000", 2024-09-15T01:33:18.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.516e-05 2024-09-15T01:33:18.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.107374+0000", 2024-09-15T01:33:18.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0174e-05 2024-09-15T01:33:18.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.109887+0000", 2024-09-15T01:33:18.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002513543 2024-09-15T01:33:18.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.539 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.539 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.539 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:18.541 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-15T01:33:18.542 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:18.539+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:18.639 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:33:18.639 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:18.641 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:18.640+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:18.647 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-15T01:33:18.742 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:18.741+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:18.743 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:18.743 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:18.743 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:33:18.743 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:33:18.743 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:18.752 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-15T01:33:18.764 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:33:18.843 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:18.842+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:18.850 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:18.850 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:33:18.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:33:18.850 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:33:18.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.850 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:33:18.850 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:33:18.850 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 62.367895363000002, 2024-09-15T01:33:18.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27210968699999999, 2024-09-15T01:33:18.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479166+0000", 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4729999999999999e-06 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479172+0000", 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7460000000000003e-06 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479186+0000", 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4011e-05 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479211+0000", 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4658e-05 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479249+0000", 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7706000000000001e-05 2024-09-15T01:33:18.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479255+0000", 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9669999999999998e-06 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750042+0000", 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27078760899999998 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750204+0000", 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016171800000000001 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750337+0000", 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000132674 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750470+0000", 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013314699999999999 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750485+0000", 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5288e-05 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750515+0000", 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0179e-05 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750582+0000", 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6890999999999994e-05 2024-09-15T01:33:18.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751212+0000", 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062971900000000005 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751249+0000", 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7072e-05 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751274+0000", 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4828999999999999e-05 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:9 3.3s0 3:c07cd22b:::smithi17529310-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.175883+0000", 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.671176258000003, 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01603094, 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:18.858 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175883+0000", 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175883+0000", 2024-09-15T01:33:18.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175884+0000", 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175886+0000", 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.429e-06 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175893+0000", 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6630000000000004e-06 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.175900+0000", 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999997e-06 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178335+0000", 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024349200000000001 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.178422+0000", 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7049e-05 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187881+0000", 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0094586740000000002 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.187983+0000", 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102005 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.191914+0000", 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039315039999999997 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 34/31 ECSubWrite(tid=13, reqid=client.4477.0:128, at_version=34'8, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.569745572000002, 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.101528777, 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.863 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7999999999999995e-07 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277326+0000", 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.184e-05 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378494+0000", 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.101167374 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378498+0000", 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1150000000000004e-06 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378565+0000", 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7354999999999995e-05 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378574+0000", 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000007e-06 2024-09-15T01:33:18.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378662+0000", 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8287000000000004e-05 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378672+0000", 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3840000000000003e-06 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378705+0000", 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3506000000000001e-05 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378722+0000", 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6449000000000001e-05 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378843+0000", 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000120937 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.247204+0000", 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.599854975, 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017667424000000001, 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247204+0000", 2024-09-15T01:33:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247204+0000", 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247205+0000", 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247219+0000", 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4110999999999999e-05 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247242+0000", 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3064000000000001e-05 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247248+0000", 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6470000000000003e-06 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247285+0000", 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7526e-05 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.247292+0000", 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4760000000000003e-06 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250915+0000", 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036228549999999999 2024-09-15T01:33:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250922+0000", 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9199999999999998e-06 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.250953+0000", 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1126e-05 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251618+0000", 2024-09-15T01:33:18.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00066501799999999997 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.264872+0000", 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013254203000000001 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.251636+0000", 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.595423119000003, 2024-09-15T01:33:18.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024080965999999999, 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251636+0000", 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251636+0000", 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251637+0000", 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7800000000000001e-07 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251655+0000", 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8062000000000001e-05 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251686+0000", 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0723999999999999e-05 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251690+0000", 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0049999999999998e-06 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251718+0000", 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8486000000000001e-05 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251725+0000", 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.252412+0000", 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068728599999999995 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.252417+0000", 2024-09-15T01:33:18.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.253911+0000", 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001493858 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.253956+0000", 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4316000000000001e-05 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275717+0000", 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021761656000000001 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:282 3.3s0 3:c4c60c95:::smithi17529310-28:head [delete] snapc 8=[8] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.256127+0000", 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.590932860000002, 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019700597, 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 282 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256127+0000", 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256127+0000", 2024-09-15T01:33:18.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256128+0000", 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3030000000000001e-06 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256132+0000", 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9990000000000002e-06 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256143+0000", 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1375e-05 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256147+0000", 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1949999999999996e-06 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256180+0000", 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2387e-05 2024-09-15T01:33:18.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256244+0000", 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4369000000000007e-05 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256731+0000", 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00048626099999999998 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256921+0000", 2024-09-15T01:33:18.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000190743 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256928+0000", 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8510000000000001e-06 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256963+0000", 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5314000000000001e-05 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256979+0000", 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5174000000000001e-05 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.258442+0000", 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00146372 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275827+0000", 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017384905999999999 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=85, reqid=osd.11.0:84, at_version=0'0, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.257015+0000", 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.590044966000001, 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019035946000000002, 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257015+0000", 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257015+0000", 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257015+0000", 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7100000000000002e-07 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257025+0000", 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0115999999999999e-05 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257047+0000", 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1645e-05 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257051+0000", 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4139999999999996e-06 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257094+0000", 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2843999999999998e-05 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257104+0000", 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.798e-06 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257241+0000", 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013668799999999999 2024-09-15T01:33:18.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257248+0000", 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5299999999999999e-06 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259196+0000", 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019477850000000001 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259787+0000", 2024-09-15T01:33:18.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00059073299999999997 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.276050+0000", 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016263822000000001 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 44/31 ECSubWrite(tid=30, reqid=osd.8.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.257430+0000", 2024-09-15T01:33:18.892 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.589629006000003, 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018688996999999999, 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257430+0000", 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257430+0000", 2024-09-15T01:33:18.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257431+0000", 2024-09-15T01:33:18.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2499999999999995e-07 2024-09-15T01:33:18.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257443+0000", 2024-09-15T01:33:18.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1728e-05 2024-09-15T01:33:18.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257465+0000", 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2013000000000001e-05 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257469+0000", 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6750000000000001e-06 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257498+0000", 2024-09-15T01:33:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8670000000000002e-05 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257503+0000", 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1019999999999996e-06 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257572+0000", 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8450999999999999e-05 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259198+0000", 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001626239 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259263+0000", 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5491e-05 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259288+0000", 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4766e-05 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.276119+0000", 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016831336999999998 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=86, reqid=osd.11.0:86, at_version=44'42, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.257959+0000", 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.589100692000002, 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018797019000000002, 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257959+0000", 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257959+0000", 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257959+0000", 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257972+0000", 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.261e-05 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257993+0000", 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1067e-05 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257996+0000", 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4690000000000002e-06 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.258029+0000", 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2691999999999998e-05 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.258035+0000", 2024-09-15T01:33:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0619999999999997e-06 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259524+0000", 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001488781 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259533+0000", 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3190000000000001e-06 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259567+0000", 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3856999999999998e-05 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259896+0000", 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032855100000000001 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.276756+0000", 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016860199999999999 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.259034+0000", 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.588025733999999, 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018774057, 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259034+0000", 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259034+0000", 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259036+0000", 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0829999999999998e-06 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259049+0000", 2024-09-15T01:33:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3283e-05 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259070+0000", 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0488000000000001e-05 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259073+0000", 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5310000000000002e-06 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259569+0000", 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049599699999999997 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259578+0000", 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7709999999999995e-06 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259766+0000", 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000187954 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259774+0000", 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1920000000000005e-06 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259800+0000", 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.635e-05 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259910+0000", 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109957 2024-09-15T01:33:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.277808+0000", 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017897451000000002 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 47/31 ECSubWriteReply(tid=162, last_complete=47'9, committed=1, applied=1))", 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:49.375678+0000", 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.471381907000001, 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.71735708600000003, 2024-09-15T01:33:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.375678+0000", 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.375678+0000", 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.375678+0000", 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2800000000000003e-07 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.375680+0000", 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.635e-06 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.092521+0000", 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.716841324 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.092538+0000", 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6983e-05 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.092633+0000", 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5618999999999994e-05 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.093035+0000", 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000401197 2024-09-15T01:33:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:509 3.2s0 3:4568dce4:::smithi17529310-45:head [copy-from ver 70] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.073110+0000", 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.773949309999999, 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.92744239399999995, 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 509 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073110+0000", 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073110+0000", 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073112+0000", 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073120+0000", 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8269999999999998e-06 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073134+0000", 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3934000000000001e-05 2024-09-15T01:33:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073142+0000", 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9899999999999997e-06 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073519+0000", 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037723500000000002 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-15T01:33:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073522+0000", 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9179999999999998e-06 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.985732+0000", 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.91221027799999999 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.985887+0000", 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015458799999999999 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.997489+0000", 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011601800000000001 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.997763+0000", 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000274586 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.997775+0000", 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1488000000000001e-05 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.998089+0000", 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031444300000000003 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.998169+0000", 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9367000000000001e-05 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.000501+0000", 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023319030000000002 2024-09-15T01:33:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.000553+0000", 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1966999999999999e-05 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:510 3.2s0 3:4568dce4:::smithi17529310-45:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.073150+0000", 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.773909922000001, 2024-09-15T01:33:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.92501677100000002, 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 510 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073150+0000", 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073150+0000", 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073150+0000", 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6300000000000005e-07 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073157+0000", 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5490000000000003e-06 2024-09-15T01:33:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073165+0000", 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.926e-06 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073169+0000", 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4429999999999996e-06 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073547+0000", 2024-09-15T01:33:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037755599999999997 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073549+0000", 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.616e-06 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.986765+0000", 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.91321570200000002 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.986800+0000", 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5169999999999997e-05 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.997953+0000", 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011152998000000001 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.998084+0000", 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013044400000000001 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.998166+0000", 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2804000000000002e-05 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:520 3.2s0 3:44cf4bb4:::smithi17529310-34:head [stat,delete] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:58.071228+0000", 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.775831201999999, 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016403424, 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 520 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.071228+0000", 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.071228+0000", 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.071230+0000", 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.739e-06 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.071236+0000", 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8130000000000001e-06 2024-09-15T01:33:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.071249+0000", 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3117e-05 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.071256+0000", 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5449999999999998e-06 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.072957+0000", 2024-09-15T01:33:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017000870000000001 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.073049+0000", 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2130000000000001e-05 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.073899+0000", 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084990499999999997 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.076533+0000", 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026339330000000002 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.078961+0000", 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00242845 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.079063+0000", 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010181199999999999 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.079342+0000", 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027918499999999999 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.079396+0000", 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4490999999999997e-05 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.087632+0000", 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082352169999999995 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.932 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 63/31 ECSubWrite(tid=46, reqid=client.4477.0:596, at_version=0'0, trim_to=0'0, roll_forward_to=63'33))", 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:05.101346+0000", 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.745713524999999, 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.96745142500000003, 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:05.101346+0000", 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:05.101346+0000", 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:05.101347+0000", 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:05.101358+0000", 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0951000000000001e-05 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068502+0000", 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.96714420499999998 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068510+0000", 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1589999999999993e-06 2024-09-15T01:33:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068576+0000", 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5767999999999998e-05 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068586+0000", 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-06 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068671+0000", 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5958000000000006e-05 2024-09-15T01:33:18.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068684+0000", 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2228999999999999e-05 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068724+0000", 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0299999999999997e-05 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068738+0000", 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3711e-05 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068797+0000", 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9648000000000002e-05 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s1 70/69 ECSubWrite(tid=70, reqid=osd.0.0:70, at_version=70'42, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:18.938 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:12.087002+0000", 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.7600574570000003, 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018043601999999999, 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.087002+0000", 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.087002+0000", 2024-09-15T01:33:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.087003+0000", 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6099999999999999e-07 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.087023+0000", 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0373e-05 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104412+0000", 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017389011999999999 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104420+0000", 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4429999999999997e-06 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104462+0000", 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2040999999999999e-05 2024-09-15T01:33:18.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104469+0000", 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3150000000000003e-06 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104729+0000", 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025947999999999998 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104754+0000", 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4949000000000001e-05 2024-09-15T01:33:18.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104793+0000", 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9610999999999997e-05 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104815+0000", 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1407999999999999e-05 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.105046+0000", 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023100900000000001 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s1 70/69 ECSubWrite(tid=72, reqid=client.4477.0:636, at_version=0'0, trim_to=0'0, roll_forward_to=70'43))", 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:12.107237+0000", 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.7398224940000002, 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.66967561099999995, 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.107237+0000", 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.107237+0000", 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.107238+0000", 2024-09-15T01:33:18.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5700000000000002e-07 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.107247+0000", 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4469999999999995e-06 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776489+0000", 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.66924175600000002 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776495+0000", 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0680000000000002e-06 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776557+0000", 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2080000000000002e-05 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776565+0000", 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5859999999999999e-06 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776657+0000", 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2614000000000005e-05 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776666+0000", 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1859999999999998e-06 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776711+0000", 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4829000000000001e-05 2024-09-15T01:33:18.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776752+0000", 2024-09-15T01:33:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0345000000000001e-05 2024-09-15T01:33:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776913+0000", 2024-09-15T01:33:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016104300000000001 2024-09-15T01:33:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:648 3.3s0 3:c07cd22b:::smithi17529310-3:head [copy-from ver 121] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.135954+0000", 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.7111056390000003, 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019156803, 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 648 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.950 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.135954+0000", 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.135954+0000", 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.135955+0000", 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4169999999999999e-06 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.135960+0000", 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4530000000000004e-06 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.135973+0000", 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2955999999999999e-05 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.135977+0000", 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2300000000000002e-06 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.136044+0000", 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7218999999999995e-05 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.136111+0000", 2024-09-15T01:33:18.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7107999999999995e-05 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.149703+0000", 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013591486999999999 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.150702+0000", 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00099899199999999998 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.150713+0000", 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1345999999999999e-05 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.151097+0000", 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038442100000000002 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.151283+0000", 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018522300000000001 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.155064+0000", 2024-09-15T01:33:18.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037815850000000001 2024-09-15T01:33:18.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.155111+0000", 2024-09-15T01:33:18.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6366000000000003e-05 2024-09-15T01:33:18.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.956 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:694 3.3s0 3:c4339fa1:::smithi17529310-16:head [copy-from ver 53] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.298369+0000", 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.5486905350000004, 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030068248999999998, 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 694 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298369+0000", 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298369+0000", 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298371+0000", 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7090000000000001e-06 2024-09-15T01:33:18.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298377+0000", 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6710000000000002e-06 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298388+0000", 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158000000000001e-05 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298394+0000", 2024-09-15T01:33:18.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2959999999999998e-06 2024-09-15T01:33:18.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298426+0000", 2024-09-15T01:33:18.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2011000000000002e-05 2024-09-15T01:33:18.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298502+0000", 2024-09-15T01:33:18.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6416999999999997e-05 2024-09-15T01:33:18.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.321468+0000", 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022965877999999999 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.321958+0000", 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049022300000000003 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.321967+0000", 2024-09-15T01:33:18.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6989999999999992e-06 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.323335+0000", 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013682259999999999 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.323352+0000", 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6782999999999998e-05 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.328391+0000", 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005038832 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.328437+0000", 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6346e-05 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:695 3.3s0 3:c4339fa1:::smithi17529310-16:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:18.963 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.298402+0000", 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.5486573720000001, 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024018102, 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 695 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:18.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298402+0000", 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298402+0000", 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298403+0000", 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298405+0000", 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8169999999999999e-06 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298409+0000", 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4220000000000002e-06 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298411+0000", 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298623+0000", 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000211516 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298679+0000", 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5652999999999997e-05 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.322176+0000", 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023497759 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.322288+0000", 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111217 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.322420+0000", 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013263399999999999 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:18.968 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:18.970 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:18.970 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:18.968+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:18.970 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:18.971 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:18.971 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:18.972 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:33:18.972 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:18.972 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:18.973 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:18.973 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:18.973 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:18.974 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:18.992 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:18.993 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:19.012 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:19.012 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:19.031 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:19.032 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:19.049 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:19.050 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:19.069 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:19.069 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:19.070 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:19.068+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:19.079 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-15T01:33:19.080 INFO:tasks.rados.rados.0.smithi175.stdout:289: done (5 left) 2024-09-15T01:33:19.080 INFO:tasks.rados.rados.0.smithi175.stdout:291: done (4 left) 2024-09-15T01:33:19.080 INFO:tasks.rados.rados.0.smithi175.stdout:292: done (3 left) 2024-09-15T01:33:19.080 INFO:tasks.rados.rados.0.smithi175.stdout:293: done (2 left) 2024-09-15T01:33:19.080 INFO:tasks.rados.rados.0.smithi175.stdout:294: done (1 left) 2024-09-15T01:33:19.081 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 7 current snap is 24 2024-09-15T01:33:19.081 INFO:tasks.rados.rados.0.smithi175.stdout:295: seq_num 133 ranges {409600=688128,1097728=720896,1818624=475136,2293760=360448} 2024-09-15T01:33:19.085 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:19.085 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:19.086 INFO:tasks.rados.rados.0.smithi175.stdout:295: writing smithi17529310-7 from 409600 to 1097728 tid 1 2024-09-15T01:33:19.091 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:19.091 INFO:tasks.rados.rados.0.smithi175.stdout:295: writing smithi17529310-7 from 1097728 to 1818624 tid 2 2024-09-15T01:33:19.094 INFO:tasks.rados.rados.0.smithi175.stdout:295: writing smithi17529310-7 from 1818624 to 2293760 tid 3 2024-09-15T01:33:19.096 INFO:tasks.rados.rados.0.smithi175.stdout:295: writing smithi17529310-7 from 2293760 to 2654208 tid 4 2024-09-15T01:33:19.096 INFO:tasks.rados.rados.0.smithi175.stdout:296: rmattr oid 32 current snap is 24 2024-09-15T01:33:19.096 INFO:tasks.rados.rados.0.smithi175.stdout:290: finishing rollback tid 0 to smithi17529310-14 2024-09-15T01:33:19.096 INFO:tasks.rados.rados.0.smithi175.stdout:290: finishing rollback tid 1 to smithi17529310-14 2024-09-15T01:33:19.097 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 14 v 59 (ObjNum 161 snap 0 seq_num 1648734544) dirty dne 2024-09-15T01:33:19.097 INFO:tasks.rados.rados.0.smithi175.stdout:295: finishing write tid 1 to smithi17529310-7 2024-09-15T01:33:19.097 INFO:tasks.rados.rados.0.smithi175.stdout:290: done (2 left) 2024-09-15T01:33:19.097 INFO:tasks.rados.rados.0.smithi175.stdout:297: read oid 8 snap 13 2024-09-15T01:33:19.097 INFO:tasks.rados.rados.0.smithi175.stdout:297: expect (ObjNum 7 snap 0 seq_num 7) 2024-09-15T01:33:19.097 INFO:tasks.rados.rados.0.smithi175.stdout:298: snap_remove snap 22 2024-09-15T01:33:19.098 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:33:19.111 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:19.111 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:19.129 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:19.130 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:19.149 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:19.150 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:19.168 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:19.169 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:19.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:19.167+0000 7f1b83fff640 0 -- 172.21.15.42:0/194015285 >> [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] conn(0x7f1b84195be0 msgr2=0x7f1b84197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:19.171 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:19.168+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:19.196 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:19.197 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:19.206 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:33:19.272 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:19.270+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:19.342 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:19.342 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:19.350 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:33:19.389 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:19.389 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:19.396 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:19.394+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:19.449 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:19.449 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:19.457 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:33:19.474 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:19.472+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:19.574 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:19.572+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:19.595 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:19.596 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:19.604 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:33:19.689 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:19.686+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:19.742 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:19.742 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:19.750 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:33:19.776 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:19.774+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:19.876 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:19.875+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:19.887 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:19.887 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:19.895 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:33:19.977 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:19.975+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:19.992 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:19.992 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:20.001 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:33:20.078 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:20.076+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:20.085 INFO:tasks.rados.rados.0.smithi175.stdout:298: done (3 left) 2024-09-15T01:33:20.085 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 16 current snap is 24 2024-09-15T01:33:20.085 INFO:tasks.rados.rados.0.smithi175.stdout:295: finishing write tid 2 to smithi17529310-7 2024-09-15T01:33:20.085 INFO:tasks.rados.rados.0.smithi175.stdout:299: seq_num 134 ranges {4145152=704512,4849664=475136,5324800=573440,5898240=507904} 2024-09-15T01:33:20.099 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:20.099 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:20.107 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-15T01:33:20.111 INFO:tasks.rados.rados.0.smithi175.stdout:299: writing smithi17529310-16 from 4145152 to 4849664 tid 1 2024-09-15T01:33:20.114 INFO:tasks.rados.rados.0.smithi175.stdout:299: writing smithi17529310-16 from 4849664 to 5324800 tid 2 2024-09-15T01:33:20.118 INFO:tasks.rados.rados.0.smithi175.stdout:299: writing smithi17529310-16 from 5324800 to 5898240 tid 3 2024-09-15T01:33:20.121 INFO:tasks.rados.rados.0.smithi175.stdout:299: writing smithi17529310-16 from 5898240 to 6406144 tid 4 2024-09-15T01:33:20.122 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 46 current snap is 24 2024-09-15T01:33:20.122 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 32 v 83 (ObjNum 86 snap 7 seq_num 86) dirty exists 2024-09-15T01:33:20.122 INFO:tasks.rados.rados.0.smithi175.stdout:295: finishing write tid 3 to smithi17529310-7 2024-09-15T01:33:20.122 INFO:tasks.rados.rados.0.smithi175.stdout:300: seq_num 135 ranges {1703936=720896,2424832=622592,3047424=688128,3735552=245760} 2024-09-15T01:33:20.135 INFO:tasks.rados.rados.0.smithi175.stdout:300: writing smithi17529310-46 from 1703936 to 2424832 tid 1 2024-09-15T01:33:20.139 INFO:tasks.rados.rados.0.smithi175.stdout:300: writing smithi17529310-46 from 2424832 to 3047424 tid 2 2024-09-15T01:33:20.143 INFO:tasks.rados.rados.0.smithi175.stdout:300: writing smithi17529310-46 from 3047424 to 3735552 tid 3 2024-09-15T01:33:20.145 INFO:tasks.rados.rados.0.smithi175.stdout:300: writing smithi17529310-46 from 3735552 to 3981312 tid 4 2024-09-15T01:33:20.145 INFO:tasks.rados.rados.0.smithi175.stdout:296: done (4 left) 2024-09-15T01:33:20.145 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 12 current snap is 24 2024-09-15T01:33:20.146 INFO:tasks.rados.rados.0.smithi175.stdout:295: finishing write tid 4 to smithi17529310-7 2024-09-15T01:33:20.146 INFO:tasks.rados.rados.0.smithi175.stdout:295: finishing write tid 5 to smithi17529310-7 2024-09-15T01:33:20.146 INFO:tasks.rados.rados.0.smithi175.stdout:297: expect (ObjNum 7 snap 0 seq_num 7) 2024-09-15T01:33:20.148 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 9] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-15T01:33:20.148 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:33:20.148 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-15T01:33:20.148 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T01:33:20.149 INFO:tasks.rados.rados.0.smithi175.stdout:295: finishing write tid 6 to smithi17529310-7 2024-09-15T01:33:20.149 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 7 v 45 (ObjNum 133 snap 24 seq_num 133) dirty exists 2024-09-15T01:33:20.149 INFO:tasks.rados.rados.0.smithi175.stdout:295: left oid 7 (ObjNum 133 snap 24 seq_num 133) 2024-09-15T01:33:20.149 INFO:tasks.rados.rados.0.smithi175.stdout:301: seq_num 136 ranges {1851392=737280,2588672=770048,3358720=540672,3899392=245760} 2024-09-15T01:33:20.162 INFO:tasks.rados.rados.0.smithi175.stdout:301: writing smithi17529310-12 from 1851392 to 2588672 tid 1 2024-09-15T01:33:20.166 INFO:tasks.rados.rados.0.smithi175.stdout:301: writing smithi17529310-12 from 2588672 to 3358720 tid 2 2024-09-15T01:33:20.169 INFO:tasks.rados.rados.0.smithi175.stdout:301: writing smithi17529310-12 from 3358720 to 3899392 tid 3 2024-09-15T01:33:20.172 INFO:tasks.rados.rados.0.smithi175.stdout:301: writing smithi17529310-12 from 3899392 to 4145152 tid 4 2024-09-15T01:33:20.172 INFO:tasks.rados.rados.0.smithi175.stdout:295: done (4 left) 2024-09-15T01:33:20.172 INFO:tasks.rados.rados.0.smithi175.stdout:297: done (3 left) 2024-09-15T01:33:20.172 INFO:tasks.rados.rados.0.smithi175.stdout:302: rollback oid 25 current snap is 24 2024-09-15T01:33:20.172 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 25 to 21 2024-09-15T01:33:20.172 INFO:tasks.rados.rados.0.smithi175.stdout:299: finishing write tid 1 to smithi17529310-16 2024-09-15T01:33:20.172 INFO:tasks.rados.rados.0.smithi175.stdout:299: finishing write tid 2 to smithi17529310-16 2024-09-15T01:33:20.172 INFO:tasks.rados.rados.0.smithi175.stdout:299: finishing write tid 3 to smithi17529310-16 2024-09-15T01:33:20.172 INFO:tasks.rados.rados.0.smithi175.stdout:299: finishing write tid 4 to smithi17529310-16 2024-09-15T01:33:20.173 INFO:tasks.rados.rados.0.smithi175.stdout:300: finishing write tid 1 to smithi17529310-46 2024-09-15T01:33:20.173 INFO:tasks.rados.rados.0.smithi175.stdout:300: finishing write tid 2 to smithi17529310-46 2024-09-15T01:33:20.173 INFO:tasks.rados.rados.0.smithi175.stdout:300: finishing write tid 3 to smithi17529310-46 2024-09-15T01:33:20.173 INFO:tasks.rados.rados.0.smithi175.stdout:300: finishing write tid 4 to smithi17529310-46 2024-09-15T01:33:20.173 INFO:tasks.rados.rados.0.smithi175.stdout:301: finishing write tid 1 to smithi17529310-12 2024-09-15T01:33:20.173 INFO:tasks.rados.rados.0.smithi175.stdout:303: setattr oid 3 current snap is 24 2024-09-15T01:33:20.173 INFO:tasks.rados.rados.0.smithi175.stdout:301: finishing write tid 2 to smithi17529310-12 2024-09-15T01:33:20.174 INFO:tasks.rados.rados.0.smithi175.stdout:304: read oid 14 snap -1 2024-09-15T01:33:20.174 INFO:tasks.rados.rados.0.smithi175.stdout:304: expect deleted 2024-09-15T01:33:20.174 INFO:tasks.rados.rados.0.smithi175.stdout:305: delete oid 41 current snap is 24 2024-09-15T01:33:20.174 INFO:tasks.rados.rados.0.smithi175.stdout:301: finishing write tid 3 to smithi17529310-12 2024-09-15T01:33:20.176 INFO:tasks.rados.rados.0.smithi175.stdout:301: finishing write tid 4 to smithi17529310-12 2024-09-15T01:33:20.176 INFO:tasks.rados.rados.0.smithi175.stdout:302: finishing rollback tid 0 to smithi17529310-25 2024-09-15T01:33:20.176 INFO:tasks.rados.rados.0.smithi175.stdout:304: done (6 left) 2024-09-15T01:33:20.176 INFO:tasks.rados.rados.0.smithi175.stdout:305: done (5 left) 2024-09-15T01:33:20.176 INFO:tasks.rados.rados.0.smithi175.stdout:306: snap_create 2024-09-15T01:33:20.176 INFO:tasks.rados.rados.0.smithi175.stdout:302: finishing rollback tid 1 to smithi17529310-25 2024-09-15T01:33:20.176 INFO:tasks.rados.rados.0.smithi175.stdout:299: finishing write tid 5 to smithi17529310-16 2024-09-15T01:33:20.179 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:20.177+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:20.214 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:20.214 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:20.223 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-15T01:33:20.281 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:20.279+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:20.359 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:20.359 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:20.367 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-15T01:33:20.411 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:20.409+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:33:20.443 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 78, 2024-09-15T01:33:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "d4803586-ed44-4154-8ee1-6b95ec3e17e7", 2024-09-15T01:33:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-15T01:32:07.285177+0000", 2024-09-15T01:33:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-15T01:33:20.077091+0000", 2024-09-15T01:33:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-15T01:32:14.443323+0000", 2024-09-15T01:33:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-15T01:33:08.068993+0000", 2024-09-15T01:33:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T01:33:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5801984, 2024-09-15T01:33:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-15T01:33:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "noscrub", 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 5, 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 16, 2024-09-15T01:33:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-15T01:33:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "luminous", 2024-09-15T01:33:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "quincy", 2024-09-15T01:33:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-15T01:33:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-15T01:33:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-15T01:33:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-15T01:32:15.118634+0000", 2024-09-15T01:33:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-15T01:33:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-15T01:33:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-15T01:33:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-15T01:33:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-15T01:33:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-15T01:33:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:33:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:33:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:33:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:33:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-15T01:33:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:33:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-15T01:33:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-15T01:33:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "24", 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:33:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:33:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-15T01:33:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:33:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-15T01:33:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-15T01:33:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-15T01:33:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-15T01:33:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-15T01:33:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-15T01:33:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-15T01:33:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-15T01:33:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-15T01:33:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-15T01:33:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-15T01:33:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-15T01:33:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-15T01:33:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:33:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:33:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:33:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-15T01:33:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-15T01:33:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-15T01:33:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-15T01:33:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-15T01:33:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-15T01:33:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-15T01:33:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-15T01:33:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:33:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:33:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:33:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:33:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-15T01:33:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-15T01:33:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-15T01:33:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-15T01:33:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-15T01:33:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-15T01:33:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-15T01:33:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-15T01:33:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-15T01:33:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-15T01:33:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-15T01:33:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-15T01:32:21.824131+0000", 2024-09-15T01:33:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-15T01:33:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T01:33:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-15T01:33:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-15T01:33:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-15T01:33:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-15T01:33:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:33:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:33:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:33:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:33:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-15T01:33:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:33:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-15T01:33:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-15T01:33:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-15T01:33:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-15T01:33:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-15T01:33:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-15T01:33:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-15T01:33:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-15T01:33:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-15T01:33:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-15T01:33:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-15T01:33:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-15T01:33:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "28", 2024-09-15T01:33:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-15T01:33:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:33:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:33:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-15T01:33:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:33:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 3, 2024-09-15T01:33:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 28, 2024-09-15T01:33:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-15T01:33:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-15T01:33:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-15T01:33:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-15T01:33:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-15T01:33:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-15T01:33:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-15T01:33:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-15T01:33:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-15T01:33:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-15T01:33:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-15T01:33:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:33:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:33:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:33:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-15T01:33:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-15T01:33:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-15T01:33:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-15T01:33:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-15T01:33:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-15T01:33:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-15T01:33:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-15T01:33:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:33:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:33:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:33:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:33:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-15T01:33:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "unique_pool_0", 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-15T01:32:32.183257+0000", 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-15T01:33:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T01:33:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 3, 2024-09-15T01:33:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 7, 2024-09-15T01:33:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 5, 2024-09-15T01:33:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 1, 2024-09-15T01:33:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:33:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:33:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:33:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:33:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-15T01:33:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:33:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 16, 2024-09-15T01:33:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 16, 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 16, 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 16, 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 16, 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "78", 2024-09-15T01:33:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-15T01:33:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:33:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:33:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-15T01:33:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:33:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 42, 2024-09-15T01:33:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 78, 2024-09-15T01:33:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-15T01:33:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-15T01:33:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-15T01:33:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-15T01:33:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-15T01:33:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "shecprofile", 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-15T01:33:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-15T01:33:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-15T01:33:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-15T01:33:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-15T01:33:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:33:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:33:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:33:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:33:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-15T01:33:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 16384, 2024-09-15T01:33:20.464 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-15T01:33:20.464 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-15T01:33:20.464 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-15T01:33:20.464 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-15T01:33:20.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rados": {} 2024-09-15T01:33:20.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.464 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:20.464 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-15T01:33:20.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.464 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-15T01:33:20.465 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "bec41583-a0a4-4712-a9ec-eec1e88fb60f", 2024-09-15T01:33:20.465 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.465 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.465 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.465 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.465 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.465 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.465 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.465 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 69, 2024-09-15T01:33:20.465 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.465 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.466 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.466 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.466 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-15T01:33:20.466 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:20.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.466 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-15T01:33:20.467 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:20.467 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.467 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.467 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.467 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.467 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-15T01:33:20.467 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:20.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.468 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.468 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-15T01:33:20.468 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:20.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.468 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.468 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.468 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.469 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.469 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-15T01:33:20.469 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:20.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.469 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.469 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-15T01:33:20.469 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:20.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.469 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.470 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.470 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.470 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-15T01:33:20.470 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:20.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.470 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-15T01:33:20.470 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28861 2024-09-15T01:33:20.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.471 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.471 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/28861", 2024-09-15T01:33:20.471 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/28861", 2024-09-15T01:33:20.471 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/28861", 2024-09-15T01:33:20.471 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/28861", 2024-09-15T01:33:20.471 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.471 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.471 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.471 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.472 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-15T01:33:20.472 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "2cca66be-8685-4a38-abf8-c864cd5434da", 2024-09-15T01:33:20.472 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.472 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.472 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.472 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.473 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.473 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.473 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.473 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 0, 2024-09-15T01:33:20.473 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.473 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.473 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.473 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.473 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.473 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6808", 2024-09-15T01:33:20.473 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:20.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.474 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6809", 2024-09-15T01:33:20.474 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:20.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.474 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.474 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.474 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.475 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6810", 2024-09-15T01:33:20.475 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:20.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.475 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6811", 2024-09-15T01:33:20.475 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:20.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.475 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.476 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.476 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.476 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6817", 2024-09-15T01:33:20.476 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:20.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.477 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6819", 2024-09-15T01:33:20.477 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:20.477 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.477 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.477 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.477 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.477 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6813", 2024-09-15T01:33:20.477 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:20.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.478 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6815", 2024-09-15T01:33:20.478 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28508 2024-09-15T01:33:20.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.478 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.478 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.115:6809/28508", 2024-09-15T01:33:20.478 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.115:6811/28508", 2024-09-15T01:33:20.478 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.115:6819/28508", 2024-09-15T01:33:20.479 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.115:6815/28508", 2024-09-15T01:33:20.479 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.479 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.479 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.479 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.479 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-15T01:33:20.479 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b8756aac-3a3e-41ba-a6ef-497063cb44df", 2024-09-15T01:33:20.479 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.479 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.479 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.480 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.480 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.480 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.480 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.480 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 66, 2024-09-15T01:33:20.480 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.480 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.480 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.480 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.481 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6808", 2024-09-15T01:33:20.481 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:20.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.481 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6809", 2024-09-15T01:33:20.481 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:20.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.481 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.481 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6810", 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6811", 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6814", 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6816", 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.483 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.484 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.484 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.484 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6812", 2024-09-15T01:33:20.484 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:20.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.484 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6813", 2024-09-15T01:33:20.485 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28732 2024-09-15T01:33:20.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.485 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.485 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.152:6809/28732", 2024-09-15T01:33:20.485 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.152:6811/28732", 2024-09-15T01:33:20.485 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.152:6816/28732", 2024-09-15T01:33:20.485 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.152:6813/28732", 2024-09-15T01:33:20.485 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.485 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.485 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.486 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.486 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-15T01:33:20.486 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "554875f5-74c0-4b58-80ee-6ba93748f104", 2024-09-15T01:33:20.486 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.486 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.486 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.486 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.486 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.486 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.486 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.487 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 75, 2024-09-15T01:33:20.487 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.487 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.487 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.487 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.487 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.487 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6802", 2024-09-15T01:33:20.487 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:20.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.487 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.488 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6803", 2024-09-15T01:33:20.488 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:20.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.488 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.488 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.488 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.488 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6804", 2024-09-15T01:33:20.488 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:20.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.489 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6805", 2024-09-15T01:33:20.489 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:20.489 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.489 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.489 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.489 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.490 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.490 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6808", 2024-09-15T01:33:20.490 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:20.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.490 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.490 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6809", 2024-09-15T01:33:20.490 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:20.490 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.490 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.490 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6806", 2024-09-15T01:33:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6807", 2024-09-15T01:33:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28835 2024-09-15T01:33:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.175:6803/28835", 2024-09-15T01:33:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.175:6805/28835", 2024-09-15T01:33:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.175:6809/28835", 2024-09-15T01:33:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.175:6807/28835", 2024-09-15T01:33:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-15T01:33:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "3fe2c05a-4c22-4dd7-a8c6-282155c82eb8", 2024-09-15T01:33:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 69, 2024-09-15T01:33:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-15T01:33:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-15T01:33:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-15T01:33:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-15T01:33:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-15T01:33:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-15T01:33:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-15T01:33:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-15T01:33:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28860 2024-09-15T01:33:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/28860", 2024-09-15T01:33:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/28860", 2024-09-15T01:33:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/28860", 2024-09-15T01:33:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/28860", 2024-09-15T01:33:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-15T01:33:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "a301d1ce-3e31-4782-ade3-f9c0f1e6a95d", 2024-09-15T01:33:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 31, 2024-09-15T01:33:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6800", 2024-09-15T01:33:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6801", 2024-09-15T01:33:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6802", 2024-09-15T01:33:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6803", 2024-09-15T01:33:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6806", 2024-09-15T01:33:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6807", 2024-09-15T01:33:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6804", 2024-09-15T01:33:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6805", 2024-09-15T01:33:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28509 2024-09-15T01:33:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.115:6801/28509", 2024-09-15T01:33:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.115:6803/28509", 2024-09-15T01:33:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.115:6807/28509", 2024-09-15T01:33:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.115:6805/28509", 2024-09-15T01:33:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-15T01:33:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "c7d180be-21c5-4674-a9f2-10a1c7794871", 2024-09-15T01:33:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 25, 2024-09-15T01:33:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6800", 2024-09-15T01:33:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6801", 2024-09-15T01:33:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6802", 2024-09-15T01:33:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6803", 2024-09-15T01:33:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6806", 2024-09-15T01:33:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6807", 2024-09-15T01:33:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6804", 2024-09-15T01:33:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6805", 2024-09-15T01:33:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28731 2024-09-15T01:33:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.152:6801/28731", 2024-09-15T01:33:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.152:6803/28731", 2024-09-15T01:33:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.152:6807/28731", 2024-09-15T01:33:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.152:6805/28731", 2024-09-15T01:33:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-15T01:33:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "4d2429ea-772f-4fb9-a421-240f1399f16c", 2024-09-15T01:33:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 31, 2024-09-15T01:33:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6810", 2024-09-15T01:33:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6811", 2024-09-15T01:33:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6812", 2024-09-15T01:33:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6813", 2024-09-15T01:33:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6816", 2024-09-15T01:33:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6817", 2024-09-15T01:33:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6814", 2024-09-15T01:33:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6815", 2024-09-15T01:33:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28836 2024-09-15T01:33:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.175:6811/28836", 2024-09-15T01:33:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.175:6813/28836", 2024-09-15T01:33:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.175:6817/28836", 2024-09-15T01:33:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.175:6815/28836", 2024-09-15T01:33:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 8, 2024-09-15T01:33:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "de7389ab-a29e-4177-9e94-484fc6a94102", 2024-09-15T01:33:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 31, 2024-09-15T01:33:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-15T01:33:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-15T01:33:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-15T01:33:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-15T01:33:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-15T01:33:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-15T01:33:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-15T01:33:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-15T01:33:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28862 2024-09-15T01:33:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/28862", 2024-09-15T01:33:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/28862", 2024-09-15T01:33:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/28862", 2024-09-15T01:33:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/28862", 2024-09-15T01:33:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 9, 2024-09-15T01:33:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "4b4b44da-97d6-41f2-8fc6-57680b10aeb8", 2024-09-15T01:33:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 0, 2024-09-15T01:33:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6812", 2024-09-15T01:33:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6814", 2024-09-15T01:33:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6816", 2024-09-15T01:33:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6818", 2024-09-15T01:33:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6822", 2024-09-15T01:33:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6823", 2024-09-15T01:33:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6820", 2024-09-15T01:33:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6821", 2024-09-15T01:33:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28510 2024-09-15T01:33:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.115:6814/28510", 2024-09-15T01:33:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.115:6818/28510", 2024-09-15T01:33:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.115:6823/28510", 2024-09-15T01:33:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.115:6821/28510", 2024-09-15T01:33:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 10, 2024-09-15T01:33:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "5f285a07-9057-4575-b911-b3977acd397c", 2024-09-15T01:33:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 0, 2024-09-15T01:33:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6815", 2024-09-15T01:33:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6817", 2024-09-15T01:33:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6818", 2024-09-15T01:33:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6819", 2024-09-15T01:33:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6822", 2024-09-15T01:33:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6824", 2024-09-15T01:33:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6820", 2024-09-15T01:33:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6821", 2024-09-15T01:33:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28733 2024-09-15T01:33:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.152:6817/28733", 2024-09-15T01:33:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.152:6819/28733", 2024-09-15T01:33:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.152:6824/28733", 2024-09-15T01:33:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.152:6821/28733", 2024-09-15T01:33:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 11, 2024-09-15T01:33:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "dacc0f3f-a598-4055-92f4-d41102ceca83", 2024-09-15T01:33:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 76, 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6826", 2024-09-15T01:33:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6827", 2024-09-15T01:33:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6828", 2024-09-15T01:33:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6829", 2024-09-15T01:33:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6832", 2024-09-15T01:33:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6833", 2024-09-15T01:33:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6830", 2024-09-15T01:33:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6831", 2024-09-15T01:33:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28837 2024-09-15T01:33:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.175:6827/28837", 2024-09-15T01:33:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.175:6829/28837", 2024-09-15T01:33:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.175:6833/28837", 2024-09-15T01:33:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.175:6831/28837", 2024-09-15T01:33:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 12, 2024-09-15T01:33:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d218197b-d0db-410c-abff-8b849a21a555", 2024-09-15T01:33:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 0, 2024-09-15T01:33:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-15T01:33:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-15T01:33:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-15T01:33:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-15T01:33:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-15T01:33:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-15T01:33:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-15T01:33:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-15T01:33:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28866 2024-09-15T01:33:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/28866", 2024-09-15T01:33:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/28866", 2024-09-15T01:33:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/28866", 2024-09-15T01:33:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/28866", 2024-09-15T01:33:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 13, 2024-09-15T01:33:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "c9bf5faf-f2ee-482e-b903-28c7718e2063", 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 54, 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6824", 2024-09-15T01:33:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6825", 2024-09-15T01:33:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6826", 2024-09-15T01:33:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6827", 2024-09-15T01:33:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6830", 2024-09-15T01:33:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6831", 2024-09-15T01:33:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6828", 2024-09-15T01:33:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.115:6829", 2024-09-15T01:33:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28511 2024-09-15T01:33:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.115:6825/28511", 2024-09-15T01:33:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.115:6827/28511", 2024-09-15T01:33:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.115:6831/28511", 2024-09-15T01:33:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.115:6829/28511", 2024-09-15T01:33:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 14, 2024-09-15T01:33:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "cc3f19cf-5a97-440b-a953-7727e6ed6a0d", 2024-09-15T01:33:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 31, 2024-09-15T01:33:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6823", 2024-09-15T01:33:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6825", 2024-09-15T01:33:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6826", 2024-09-15T01:33:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6827", 2024-09-15T01:33:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6830", 2024-09-15T01:33:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6831", 2024-09-15T01:33:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6828", 2024-09-15T01:33:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.152:6829", 2024-09-15T01:33:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28734 2024-09-15T01:33:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.152:6825/28734", 2024-09-15T01:33:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.152:6827/28734", 2024-09-15T01:33:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.152:6831/28734", 2024-09-15T01:33:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.152:6829/28734", 2024-09-15T01:33:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 15, 2024-09-15T01:33:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "51290c23-e86d-4e05-9ac8-ca53d6a1eb80", 2024-09-15T01:33:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-15T01:33:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-15T01:33:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-15T01:33:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-15T01:33:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-15T01:33:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-15T01:33:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 21, 2024-09-15T01:33:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 31, 2024-09-15T01:33:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-15T01:33:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-15T01:33:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-15T01:33:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6818", 2024-09-15T01:33:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6819", 2024-09-15T01:33:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-15T01:33:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6820", 2024-09-15T01:33:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6821", 2024-09-15T01:33:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-15T01:33:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6824", 2024-09-15T01:33:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6825", 2024-09-15T01:33:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-15T01:33:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-15T01:33:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-15T01:33:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6822", 2024-09-15T01:33:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-15T01:33:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.175:6823", 2024-09-15T01:33:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 28838 2024-09-15T01:33:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.175:6819/28838", 2024-09-15T01:33:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.175:6821/28838", 2024-09-15T01:33:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.175:6825/28838", 2024-09-15T01:33:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.175:6823/28838", 2024-09-15T01:33:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-15T01:33:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-15T01:33:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-15T01:33:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-15T01:33:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-15T01:33:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:33:15.503649+0000", 2024-09-15T01:33:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-15T01:33:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.544884+0000", 2024-09-15T01:33:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-15T01:33:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.614659+0000", 2024-09-15T01:33:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-15T01:33:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.605046+0000", 2024-09-15T01:33:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-15T01:33:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.503545+0000", 2024-09-15T01:33:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-15T01:33:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.584788+0000", 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.563649+0000", 2024-09-15T01:33:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-15T01:33:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.622071+0000", 2024-09-15T01:33:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 8, 2024-09-15T01:33:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.572859+0000", 2024-09-15T01:33:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 9, 2024-09-15T01:33:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:33:13.348208+0000", 2024-09-15T01:33:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 10, 2024-09-15T01:33:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.534845+0000", 2024-09-15T01:33:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 11, 2024-09-15T01:33:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.556801+0000", 2024-09-15T01:33:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 12, 2024-09-15T01:33:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.565526+0000", 2024-09-15T01:33:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 13, 2024-09-15T01:33:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.582172+0000", 2024-09-15T01:33:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 14, 2024-09-15T01:33:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:33:19.554554+0000", 2024-09-15T01:33:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 15, 2024-09-15T01:33:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-15T01:33:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-15T01:33:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-15T01:33:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-15T01:33:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-15T01:33:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-15T01:32:12.619890+0000", 2024-09-15T01:33:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-15T01:33:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [ 2024-09-15T01:33:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.9", 2024-09-15T01:33:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-15T01:33:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: 11, 2024-09-15T01:33:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: 3, 2024-09-15T01:33:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: 6, 2024-09-15T01:33:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-09-15T01:33:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: 13, 2024-09-15T01:33:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: 5, 2024-09-15T01:33:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: 9 2024-09-15T01:33:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-15T01:33:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-15T01:33:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-15T01:33:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-15T01:33:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-15T01:33:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-15T01:33:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-15T01:33:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-15T01:33:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-15T01:33:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-15T01:33:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-15T01:33:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-15T01:33:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "shecprofile": { 2024-09-15T01:33:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "c": "2", 2024-09-15T01:33:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-device-class": "", 2024-09-15T01:33:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-15T01:33:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-root": "default", 2024-09-15T01:33:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "4", 2024-09-15T01:33:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "3", 2024-09-15T01:33:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "shecprofile", 2024-09-15T01:33:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "shec", 2024-09-15T01:33:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "multiple" 2024-09-15T01:33:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [ 2024-09-15T01:33:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-15T01:33:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-15T01:33:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 21, 2024-09-15T01:33:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-15T01:33:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 29, 2024-09-15T01:33:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 2 2024-09-15T01:33:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 33, 2024-09-15T01:33:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-15T01:33:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 39, 2024-09-15T01:33:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-15T01:33:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [ 2024-09-15T01:33:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-15T01:33:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-15T01:33:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 39, 2024-09-15T01:33:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-15T01:33:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-15T01:33:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-15T01:33:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-15T01:33:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-15T01:33:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-15T01:33:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-15T01:33:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-15T01:33:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-15T01:33:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-15T01:33:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:20.595 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:20.597 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:20.597 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:20.598 INFO:tasks.thrashosds.thrasher:j is {'epoch': 78, 'fsid': 'd4803586-ed44-4154-8ee1-6b95ec3e17e7', 'created': '2024-09-15T01:32:07.285177+0000', 'modified': '2024-09-15T01:33:20.077091+0000', 'last_up_change': '2024-09-15T01:32:14.443323+0000', 'last_in_change': '2024-09-15T01:33:08.068993+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T01:32:15.118634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T01:32:21.824131+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 28, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-15T01:32:32.183257+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 42, 'snap_epoch': 78, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'bec41583-a0a4-4712-a9ec-eec1e88fb60f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 28861}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 28861}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 28861}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 28861}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 28861}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 28861}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 28861}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 28861}]}, 'public_addr': '172.21.15.42:6809/28861', 'cluster_addr': '172.21.15.42:6811/28861', 'heartbeat_back_addr': '172.21.15.42:6815/28861', 'heartbeat_front_addr': '172.21.15.42:6813/28861', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '2cca66be-8685-4a38-abf8-c864cd5434da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 28508}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 28508}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 28508}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 28508}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6817', 'nonce': 28508}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 28508}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6813', 'nonce': 28508}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 28508}]}, 'public_addr': '172.21.15.115:6809/28508', 'cluster_addr': '172.21.15.115:6811/28508', 'heartbeat_back_addr': '172.21.15.115:6819/28508', 'heartbeat_front_addr': '172.21.15.115:6815/28508', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'b8756aac-3a3e-41ba-a6ef-497063cb44df', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6808', 'nonce': 28732}, {'type': 'v1', 'addr': '172.21.15.152:6809', 'nonce': 28732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6810', 'nonce': 28732}, {'type': 'v1', 'addr': '172.21.15.152:6811', 'nonce': 28732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6814', 'nonce': 28732}, {'type': 'v1', 'addr': '172.21.15.152:6816', 'nonce': 28732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6812', 'nonce': 28732}, {'type': 'v1', 'addr': '172.21.15.152:6813', 'nonce': 28732}]}, 'public_addr': '172.21.15.152:6809/28732', 'cluster_addr': '172.21.15.152:6811/28732', 'heartbeat_back_addr': '172.21.15.152:6816/28732', 'heartbeat_front_addr': '172.21.15.152:6813/28732', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '554875f5-74c0-4b58-80ee-6ba93748f104', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6802', 'nonce': 28835}, {'type': 'v1', 'addr': '172.21.15.175:6803', 'nonce': 28835}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6804', 'nonce': 28835}, {'type': 'v1', 'addr': '172.21.15.175:6805', 'nonce': 28835}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6808', 'nonce': 28835}, {'type': 'v1', 'addr': '172.21.15.175:6809', 'nonce': 28835}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6806', 'nonce': 28835}, {'type': 'v1', 'addr': '172.21.15.175:6807', 'nonce': 28835}]}, 'public_addr': '172.21.15.175:6803/28835', 'cluster_addr': '172.21.15.175:6805/28835', 'heartbeat_back_addr': '172.21.15.175:6809/28835', 'heartbeat_front_addr': '172.21.15.175:6807/28835', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3fe2c05a-4c22-4dd7-a8c6-282155c82eb8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 28860}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 28860}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 28860}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 28860}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 28860}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 28860}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 28860}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 28860}]}, 'public_addr': '172.21.15.42:6801/28860', 'cluster_addr': '172.21.15.42:6803/28860', 'heartbeat_back_addr': '172.21.15.42:6807/28860', 'heartbeat_front_addr': '172.21.15.42:6805/28860', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a301d1ce-3e31-4782-ade3-f9c0f1e6a95d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 28509}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 28509}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 28509}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 28509}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 28509}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 28509}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 28509}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 28509}]}, 'public_addr': '172.21.15.115:6801/28509', 'cluster_addr': '172.21.15.115:6803/28509', 'heartbeat_back_addr': '172.21.15.115:6807/28509', 'heartbeat_front_addr': '172.21.15.115:6805/28509', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c7d180be-21c5-4674-a9f2-10a1c7794871', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6800', 'nonce': 28731}, {'type': 'v1', 'addr': '172.21.15.152:6801', 'nonce': 28731}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6802', 'nonce': 28731}, {'type': 'v1', 'addr': '172.21.15.152:6803', 'nonce': 28731}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6806', 'nonce': 28731}, {'type': 'v1', 'addr': '172.21.15.152:6807', 'nonce': 28731}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6804', 'nonce': 28731}, {'type': 'v1', 'addr': '172.21.15.152:6805', 'nonce': 28731}]}, 'public_addr': '172.21.15.152:6801/28731', 'cluster_addr': '172.21.15.152:6803/28731', 'heartbeat_back_addr': '172.21.15.152:6807/28731', 'heartbeat_front_addr': '172.21.15.152:6805/28731', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4d2429ea-772f-4fb9-a421-240f1399f16c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6810', 'nonce': 28836}, {'type': 'v1', 'addr': '172.21.15.175:6811', 'nonce': 28836}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6812', 'nonce': 28836}, {'type': 'v1', 'addr': '172.21.15.175:6813', 'nonce': 28836}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6816', 'nonce': 28836}, {'type': 'v1', 'addr': '172.21.15.175:6817', 'nonce': 28836}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6814', 'nonce': 28836}, {'type': 'v1', 'addr': '172.21.15.175:6815', 'nonce': 28836}]}, 'public_addr': '172.21.15.175:6811/28836', 'cluster_addr': '172.21.15.175:6813/28836', 'heartbeat_back_addr': '172.21.15.175:6817/28836', 'heartbeat_front_addr': '172.21.15.175:6815/28836', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': 'de7389ab-a29e-4177-9e94-484fc6a94102', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 28862}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 28862}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 28862}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 28862}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 28862}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 28862}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 28862}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 28862}]}, 'public_addr': '172.21.15.42:6817/28862', 'cluster_addr': '172.21.15.42:6819/28862', 'heartbeat_back_addr': '172.21.15.42:6823/28862', 'heartbeat_front_addr': '172.21.15.42:6821/28862', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '4b4b44da-97d6-41f2-8fc6-57680b10aeb8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 28510}, {'type': 'v1', 'addr': '172.21.15.115:6814', 'nonce': 28510}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 28510}, {'type': 'v1', 'addr': '172.21.15.115:6818', 'nonce': 28510}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 28510}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 28510}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 28510}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 28510}]}, 'public_addr': '172.21.15.115:6814/28510', 'cluster_addr': '172.21.15.115:6818/28510', 'heartbeat_back_addr': '172.21.15.115:6823/28510', 'heartbeat_front_addr': '172.21.15.115:6821/28510', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '5f285a07-9057-4575-b911-b3977acd397c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6815', 'nonce': 28733}, {'type': 'v1', 'addr': '172.21.15.152:6817', 'nonce': 28733}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6818', 'nonce': 28733}, {'type': 'v1', 'addr': '172.21.15.152:6819', 'nonce': 28733}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6822', 'nonce': 28733}, {'type': 'v1', 'addr': '172.21.15.152:6824', 'nonce': 28733}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6820', 'nonce': 28733}, {'type': 'v1', 'addr': '172.21.15.152:6821', 'nonce': 28733}]}, 'public_addr': '172.21.15.152:6817/28733', 'cluster_addr': '172.21.15.152:6819/28733', 'heartbeat_back_addr': '172.21.15.152:6824/28733', 'heartbeat_front_addr': '172.21.15.152:6821/28733', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'dacc0f3f-a598-4055-92f4-d41102ceca83', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6826', 'nonce': 28837}, {'type': 'v1', 'addr': '172.21.15.175:6827', 'nonce': 28837}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6828', 'nonce': 28837}, {'type': 'v1', 'addr': '172.21.15.175:6829', 'nonce': 28837}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 28837}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 28837}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6830', 'nonce': 28837}, {'type': 'v1', 'addr': '172.21.15.175:6831', 'nonce': 28837}]}, 'public_addr': '172.21.15.175:6827/28837', 'cluster_addr': '172.21.15.175:6829/28837', 'heartbeat_back_addr': '172.21.15.175:6833/28837', 'heartbeat_front_addr': '172.21.15.175:6831/28837', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': 'd218197b-d0db-410c-abff-8b849a21a555', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 28866}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 28866}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 28866}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 28866}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 28866}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 28866}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 28866}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 28866}]}, 'public_addr': '172.21.15.42:6825/28866', 'cluster_addr': '172.21.15.42:6827/28866', 'heartbeat_back_addr': '172.21.15.42:6831/28866', 'heartbeat_front_addr': '172.21.15.42:6829/28866', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': 'c9bf5faf-f2ee-482e-b903-28c7718e2063', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 28511}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 28511}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 28511}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 28511}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 28511}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 28511}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 28511}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 28511}]}, 'public_addr': '172.21.15.115:6825/28511', 'cluster_addr': '172.21.15.115:6827/28511', 'heartbeat_back_addr': '172.21.15.115:6831/28511', 'heartbeat_front_addr': '172.21.15.115:6829/28511', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': 'cc3f19cf-5a97-440b-a953-7727e6ed6a0d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6823', 'nonce': 28734}, {'type': 'v1', 'addr': '172.21.15.152:6825', 'nonce': 28734}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6826', 'nonce': 28734}, {'type': 'v1', 'addr': '172.21.15.152:6827', 'nonce': 28734}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6830', 'nonce': 28734}, {'type': 'v1', 'addr': '172.21.15.152:6831', 'nonce': 28734}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6828', 'nonce': 28734}, {'type': 'v1', 'addr': '172.21.15.152:6829', 'nonce': 28734}]}, 'public_addr': '172.21.15.152:6825/28734', 'cluster_addr': '172.21.15.152:6827/28734', 'heartbeat_back_addr': '172.21.15.152:6831/28734', 'heartbeat_front_addr': '172.21.15.152:6829/28734', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '51290c23-e86d-4e05-9ac8-ca53d6a1eb80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6818', 'nonce': 28838}, {'type': 'v1', 'addr': '172.21.15.175:6819', 'nonce': 28838}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6820', 'nonce': 28838}, {'type': 'v1', 'addr': '172.21.15.175:6821', 'nonce': 28838}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6824', 'nonce': 28838}, {'type': 'v1', 'addr': '172.21.15.175:6825', 'nonce': 28838}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6822', 'nonce': 28838}, {'type': 'v1', 'addr': '172.21.15.175:6823', 'nonce': 28838}]}, 'public_addr': '172.21.15.175:6819/28838', 'cluster_addr': '172.21.15.175:6821/28838', 'heartbeat_back_addr': '172.21.15.175:6825/28838', 'heartbeat_front_addr': '172.21.15.175:6823/28838', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:33:15.503649+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.544884+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.614659+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.605046+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.503545+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.584788+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.563649+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.622071+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.572859+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:33:13.348208+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.534845+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.556801+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.565526+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.582172+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:33:19.554554+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:32:12.619890+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.9', 'osds': [11, 3, 6, 0, 13, 5, 9]}], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 21, 'length': 1}, {'begin': 29, 'length': 2}, {'begin': 33, 'length': 1}, {'begin': 39, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 39, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T01:33:20.598 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:33:20.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:33:20.600 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:20.594+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:20.602 INFO:tasks.rados.rados.0.smithi175.stdout:300: finishing write tid 5 to smithi17529310-46 2024-09-15T01:33:20.603 INFO:tasks.rados.rados.0.smithi175.stdout:300: finishing write tid 6 to smithi17529310-46 2024-09-15T01:33:20.603 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 46 v 95 (ObjNum 135 snap 24 seq_num 135) dirty exists 2024-09-15T01:33:20.603 INFO:tasks.rados.rados.0.smithi175.stdout:300: left oid 46 (ObjNum 135 snap 24 seq_num 135) 2024-09-15T01:33:20.697 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:20.695+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:20.702 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:20.702 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:20.710 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:33:20.734 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 3 v 132 (ObjNum 118 snap 19 seq_num 118) dirty exists 2024-09-15T01:33:20.735 INFO:tasks.rados.rados.0.smithi175.stdout:299: finishing write tid 6 to smithi17529310-16 2024-09-15T01:33:20.735 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 16 v 131 (ObjNum 134 snap 24 seq_num 134) dirty exists 2024-09-15T01:33:20.735 INFO:tasks.rados.rados.0.smithi175.stdout:299: left oid 16 (ObjNum 134 snap 24 seq_num 134) 2024-09-15T01:33:20.798 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:20.796+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:20.830 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:20.830 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:20.839 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:33:20.899 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:20.898+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:20.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:20.903+0000 7f2699539640 0 --2- 172.21.15.42:0/3329556950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26940c0d00 0x7f2694166250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T01:33:20.929 INFO:tasks.rados.rados.0.smithi175.stdout:302: finishing rollback tid 2 to smithi17529310-25 2024-09-15T01:33:20.930 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 25 v 51 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-15T01:33:20.933 INFO:tasks.rados.rados.0.smithi175.stdout:301: finishing write tid 5 to smithi17529310-12 2024-09-15T01:33:20.934 INFO:tasks.rados.rados.0.smithi175.stdout:301: finishing write tid 6 to smithi17529310-12 2024-09-15T01:33:20.934 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 12 v 91 (ObjNum 136 snap 24 seq_num 136) dirty exists 2024-09-15T01:33:20.934 INFO:tasks.rados.rados.0.smithi175.stdout:301: left oid 12 (ObjNum 136 snap 24 seq_num 136) 2024-09-15T01:33:20.976 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:20.976 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:20.984 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:33:21.028 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:21.026+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:21.077 INFO:tasks.rados.rados.0.smithi175.stdout:299: done (5 left) 2024-09-15T01:33:21.077 INFO:tasks.rados.rados.0.smithi175.stdout:300: done (4 left) 2024-09-15T01:33:21.077 INFO:tasks.rados.rados.0.smithi175.stdout:301: done (3 left) 2024-09-15T01:33:21.077 INFO:tasks.rados.rados.0.smithi175.stdout:302: done (2 left) 2024-09-15T01:33:21.077 INFO:tasks.rados.rados.0.smithi175.stdout:303: done (1 left) 2024-09-15T01:33:21.077 INFO:tasks.rados.rados.0.smithi175.stdout:306: done (0 left) 2024-09-15T01:33:21.077 INFO:tasks.rados.rados.0.smithi175.stdout:307: snap_create 2024-09-15T01:33:21.081 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:21.081 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:21.085 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:33:21.086 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:33:21.090 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:33:21.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T01:33:21.098 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-09-15T01:33:21.070433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":122634803,"num_objects":96,"num_object_clones":49,"num_object_copies":652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":6,"num_read":410,"num_read_kb":96010,"num_write":878,"num_write_kb":155202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":44957696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1098,"ondisk_log_size":1098,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":16,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":13157604,"kb_used_data":1485735268,"kb_used_omap":32201,"kb_used_meta":0,"kb_avail":1485716252,"statfs":{"total":1534846828544,"available":1521373442048,"internally_reserved":19472384,"allocated":1521392914432,"data_stored":1521392914432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32974050,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,2,0,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":19,"commit_latency_ns":0,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8650752,"num_objects":11,"num_object_clones":11,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":6,"num_read_kb":386,"num_write":71,"num_write_kb":8448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.999344"},"pg_stats":[{"pgid":"3.f","version":"76'165","reported_seq":287,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T01:33:18.106932+0000","last_change":"2024-09-15T01:33:18.106932+0000","last_active":"2024-09-15T01:33:18.106932+0000","last_peered":"2024-09-15T01:33:18.106932+0000","last_clean":"2024-09-15T01:33:18.106932+0000","last_became_active":"2024-09-15T01:32:34.487273+0000","last_became_peered":"2024-09-15T01:32:34.487273+0000","last_unstale":"2024-09-15T01:33:18.106932+0000","last_undegraded":"2024-09-15T01:33:18.106932+0000","last_fullsized":"2024-09-15T01:33:18.106932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:42.230488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026953080000000001,"stat_sum":{"num_bytes":22822912,"num_objects":16,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":35,"num_read_kb":295,"num_write":129,"num_write_kb":27520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":173,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T01:33:20.115419+0000","last_change":"2024-09-15T01:33:20.115673+0000","last_active":"2024-09-15T01:33:20.115419+0000","last_peered":"2024-09-15T01:33:20.115419+0000","last_clean":"2024-09-15T01:33:20.115419+0000","last_became_active":"2024-09-15T01:33:09.327259+0000","last_became_peered":"2024-09-15T01:33:09.327259+0000","last_unstale":"2024-09-15T01:33:20.115419+0000","last_undegraded":"2024-09-15T01:33:20.115419+0000","last_fullsized":"2024-09-15T01:33:20.115419+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:57.127309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037856599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,1,11,8,6,12],"acting":[2,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"},{"start":"27","length":"1"}]},{"pgid":"3.d","version":"76'75","reported_seq":207,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T01:33:20.083052+0000","last_change":"2024-09-15T01:33:20.083374+0000","last_active":"2024-09-15T01:33:20.083052+0000","last_peered":"2024-09-15T01:33:20.083052+0000","last_clean":"2024-09-15T01:33:20.083052+0000","last_became_active":"2024-09-15T01:32:34.487040+0000","last_became_peered":"2024-09-15T01:32:34.487040+0000","last_unstale":"2024-09-15T01:33:20.083052+0000","last_undegraded":"2024-09-15T01:33:20.083052+0000","last_fullsized":"2024-09-15T01:33:20.083052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.966217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042093300000000002,"stat_sum":{"num_bytes":7356416,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":20,"num_read_kb":873,"num_write":51,"num_write_kb":8240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"},{"start":"27","length":"1"}]},{"pgid":"3.c","version":"76'89","reported_seq":222,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T01:33:18.094170+0000","last_change":"2024-09-15T01:33:18.094170+0000","last_active":"2024-09-15T01:33:18.094170+0000","last_peered":"2024-09-15T01:33:18.094170+0000","last_clean":"2024-09-15T01:33:18.094170+0000","last_became_active":"2024-09-15T01:32:34.487072+0000","last_became_peered":"2024-09-15T01:32:34.487072+0000","last_unstale":"2024-09-15T01:33:18.094170+0000","last_undegraded":"2024-09-15T01:33:18.094170+0000","last_fullsized":"2024-09-15T01:33:18.094170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.186197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014199514999999999,"stat_sum":{"num_bytes":13303808,"num_objects":9,"num_object_clones":5,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":26,"num_read_kb":17150,"num_write":63,"num_write_kb":14080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"}]},{"pgid":"3.b","version":"76'33","reported_seq":240,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T01:33:20.116501+0000","last_change":"2024-09-15T01:33:20.116900+0000","last_active":"2024-09-15T01:33:20.116501+0000","last_peered":"2024-09-15T01:33:20.116501+0000","last_clean":"2024-09-15T01:33:20.116501+0000","last_became_active":"2024-09-15T01:33:12.080816+0000","last_became_peered":"2024-09-15T01:33:12.080816+0000","last_unstale":"2024-09-15T01:33:20.116501+0000","last_undegraded":"2024-09-15T01:33:20.116501+0000","last_fullsized":"2024-09-15T01:33:20.116501+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:58.058984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113743,"stat_sum":{"num_bytes":4603904,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":15,"num_write_kb":6640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":3538944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"},{"start":"27","length":"1"}]},{"pgid":"3.a","version":"76'39","reported_seq":283,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T01:33:20.115960+0000","last_change":"2024-09-15T01:33:20.116377+0000","last_active":"2024-09-15T01:33:20.115960+0000","last_peered":"2024-09-15T01:33:20.115960+0000","last_clean":"2024-09-15T01:33:20.115960+0000","last_became_active":"2024-09-15T01:33:12.077513+0000","last_became_peered":"2024-09-15T01:33:12.077513+0000","last_unstale":"2024-09-15T01:33:20.115960+0000","last_undegraded":"2024-09-15T01:33:20.115960+0000","last_fullsized":"2024-09-15T01:33:20.115960+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:58.853412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086131900000000004,"stat_sum":{"num_bytes":3457024,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":9,"num_read_kb":3,"num_write":16,"num_write_kb":5584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":7471104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"},{"start":"27","length":"1"}]},{"pgid":"3.9","version":"77'59","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T01:33:19.099872+0000","last_change":"2024-09-15T01:33:19.090218+0000","last_active":"2024-09-15T01:33:19.099872+0000","last_peered":"2024-09-15T01:33:19.099872+0000","last_clean":"2024-09-15T01:33:19.099872+0000","last_became_active":"2024-09-15T01:33:19.083379+0000","last_became_peered":"2024-09-15T01:33:19.083379+0000","last_unstale":"2024-09-15T01:33:19.099872+0000","last_undegraded":"2024-09-15T01:33:19.099872+0000","last_fullsized":"2024-09-15T01:33:19.099872+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:58.063827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042963089999999999,"stat_sum":{"num_bytes":1277952,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":26,"num_read_kb":2987,"num_write":30,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6,0,13,5,9],"acting":[11,3,6,0,13,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"}]},{"pgid":"3.8","version":"78'91","reported_seq":224,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T01:33:20.173136+0000","last_change":"2024-09-15T01:33:20.122394+0000","last_active":"2024-09-15T01:33:20.173136+0000","last_peered":"2024-09-15T01:33:20.173136+0000","last_clean":"2024-09-15T01:33:20.173136+0000","last_became_active":"2024-09-15T01:32:34.190371+0000","last_became_peered":"2024-09-15T01:32:34.190371+0000","last_unstale":"2024-09-15T01:33:20.173136+0000","last_undegraded":"2024-09-15T01:33:20.173136+0000","last_fullsized":"2024-09-15T01:33:20.173136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:51.142885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068167180000000003,"stat_sum":{"num_bytes":13058048,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":20,"num_read_kb":15850,"num_write":86,"num_write_kb":13440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"},{"start":"27","length":"1"}]},{"pgid":"1.0","version":"24'112","reported_seq":198,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T01:33:19.080540+0000","last_change":"2024-09-15T01:32:16.749959+0000","last_active":"2024-09-15T01:33:19.080540+0000","last_peered":"2024-09-15T01:33:19.080540+0000","last_clean":"2024-09-15T01:33:19.080540+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:33:19.080540+0000","last_undegraded":"2024-09-15T01:33:19.080540+0000","last_fullsized":"2024-09-15T01:33:19.080540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_clean_scrub_stamp":"2024-09-15T01:32:15.451214+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T01:33:19.078103+0000","last_change":"2024-09-15T01:32:25.147954+0000","last_active":"2024-09-15T01:33:19.078103+0000","last_peered":"2024-09-15T01:33:19.078103+0000","last_clean":"2024-09-15T01:33:19.078103+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:33:19.078103+0000","last_undegraded":"2024-09-15T01:33:19.078103+0000","last_fullsized":"2024-09-15T01:33:19.078103+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.228776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"73'44","reported_seq":220,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T01:33:20.113016+0000","last_change":"2024-09-15T01:33:20.113325+0000","last_active":"2024-09-15T01:33:20.113016+0000","last_peered":"2024-09-15T01:33:20.113016+0000","last_clean":"2024-09-15T01:33:20.113016+0000","last_became_active":"2024-09-15T01:32:57.985433+0000","last_became_peered":"2024-09-15T01:32:57.985433+0000","last_unstale":"2024-09-15T01:33:20.113016+0000","last_undegraded":"2024-09-15T01:33:20.113016+0000","last_fullsized":"2024-09-15T01:33:20.113016+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:55.388361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044329900000000003,"stat_sum":{"num_bytes":3686400,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":17,"num_read_kb":3847,"num_write":26,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":5652480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"},{"start":"27","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T01:33:19.080503+0000","last_change":"2024-09-15T01:32:25.143709+0000","last_active":"2024-09-15T01:33:19.080503+0000","last_peered":"2024-09-15T01:33:19.080503+0000","last_clean":"2024-09-15T01:33:19.080503+0000","last_became_active":"2024-09-15T01:32:23.134382+0000","last_became_peered":"2024-09-15T01:32:23.134382+0000","last_unstale":"2024-09-15T01:33:19.080503+0000","last_undegraded":"2024-09-15T01:33:19.080503+0000","last_fullsized":"2024-09-15T01:33:19.080503+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.326224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"76'23","reported_seq":266,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T01:33:20.115693+0000","last_change":"2024-09-15T01:33:20.116425+0000","last_active":"2024-09-15T01:33:20.115693+0000","last_peered":"2024-09-15T01:33:20.115693+0000","last_clean":"2024-09-15T01:33:20.115693+0000","last_became_active":"2024-09-15T01:33:12.080783+0000","last_became_peered":"2024-09-15T01:33:12.080783+0000","last_unstale":"2024-09-15T01:33:20.115693+0000","last_undegraded":"2024-09-15T01:33:20.115693+0000","last_fullsized":"2024-09-15T01:33:20.115693+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:39.907988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089577899999999998,"stat_sum":{"num_bytes":1867776,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":11,"num_read_kb":1059,"num_write":11,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":4177920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,2,13],"acting":[4,11,9,5,10,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"},{"start":"27","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T01:33:19.081058+0000","last_change":"2024-09-15T01:32:25.583876+0000","last_active":"2024-09-15T01:33:19.081058+0000","last_peered":"2024-09-15T01:33:19.081058+0000","last_clean":"2024-09-15T01:33:19.081058+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:33:19.081058+0000","last_undegraded":"2024-09-15T01:33:19.081058+0000","last_fullsized":"2024-09-15T01:33:19.081058+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:49.092006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"78'51","reported_seq":192,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T01:33:20.175116+0000","last_change":"2024-09-15T01:33:20.122558+0000","last_active":"2024-09-15T01:33:20.175116+0000","last_peered":"2024-09-15T01:33:20.175116+0000","last_clean":"2024-09-15T01:33:20.175116+0000","last_became_active":"2024-09-15T01:32:34.190384+0000","last_became_peered":"2024-09-15T01:32:34.190384+0000","last_unstale":"2024-09-15T01:33:20.175116+0000","last_undegraded":"2024-09-15T01:33:20.175116+0000","last_fullsized":"2024-09-15T01:33:20.175116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.588055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045442089999999996,"stat_sum":{"num_bytes":3063808,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":3145,"num_write":23,"num_write_kb":5024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"},{"start":"27","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":67,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T01:33:19.080792+0000","last_change":"2024-09-15T01:32:25.144113+0000","last_active":"2024-09-15T01:33:19.080792+0000","last_peered":"2024-09-15T01:33:19.080792+0000","last_clean":"2024-09-15T01:33:19.080792+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:33:19.080792+0000","last_undegraded":"2024-09-15T01:33:19.080792+0000","last_fullsized":"2024-09-15T01:33:19.080792+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"78'74","reported_seq":212,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T01:33:20.135960+0000","last_change":"2024-09-15T01:33:20.115412+0000","last_active":"2024-09-15T01:33:20.135960+0000","last_peered":"2024-09-15T01:33:20.135960+0000","last_clean":"2024-09-15T01:33:20.135960+0000","last_became_active":"2024-09-15T01:32:34.170218+0000","last_became_peered":"2024-09-15T01:32:34.170218+0000","last_unstale":"2024-09-15T01:33:20.135960+0000","last_undegraded":"2024-09-15T01:33:20.135960+0000","last_fullsized":"2024-09-15T01:33:20.135960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.093067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024288249999999999,"stat_sum":{"num_bytes":11993088,"num_objects":7,"num_object_clones":4,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":1546,"num_write":77,"num_write_kb":15056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"},{"start":"27","length":"1"}]},{"pgid":"3.5","version":"76'56","reported_seq":188,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T01:33:18.093384+0000","last_change":"2024-09-15T01:33:18.093384+0000","last_active":"2024-09-15T01:33:18.093384+0000","last_peered":"2024-09-15T01:33:18.093384+0000","last_clean":"2024-09-15T01:33:18.093384+0000","last_became_active":"2024-09-15T01:32:34.189813+0000","last_became_peered":"2024-09-15T01:32:34.189813+0000","last_unstale":"2024-09-15T01:33:18.093384+0000","last_undegraded":"2024-09-15T01:33:18.093384+0000","last_fullsized":"2024-09-15T01:33:18.093384+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:47.814751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0092957779999999993,"stat_sum":{"num_bytes":8306688,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":20,"num_read_kb":6009,"num_write":27,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":64,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T01:33:19.078314+0000","last_change":"2024-09-15T01:32:25.568260+0000","last_active":"2024-09-15T01:33:19.078314+0000","last_peered":"2024-09-15T01:33:19.078314+0000","last_clean":"2024-09-15T01:33:19.078314+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:33:19.078314+0000","last_undegraded":"2024-09-15T01:33:19.078314+0000","last_fullsized":"2024-09-15T01:33:19.078314+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:32.165806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"78'63","reported_seq":200,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T01:33:20.120489+0000","last_change":"2024-09-15T01:33:20.120489+0000","last_active":"2024-09-15T01:33:20.120489+0000","last_peered":"2024-09-15T01:33:20.120489+0000","last_clean":"2024-09-15T01:33:20.120489+0000","last_became_active":"2024-09-15T01:32:34.189644+0000","last_became_peered":"2024-09-15T01:32:34.189644+0000","last_unstale":"2024-09-15T01:33:20.120489+0000","last_undegraded":"2024-09-15T01:33:20.120489+0000","last_fullsized":"2024-09-15T01:33:20.120489+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:02.903663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050607699999999996,"stat_sum":{"num_bytes":6356992,"num_objects":7,"num_object_clones":4,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":13067,"num_write":66,"num_write_kb":8432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"},{"start":"27","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":64,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T01:33:19.080783+0000","last_change":"2024-09-15T01:32:25.407396+0000","last_active":"2024-09-15T01:33:19.080783+0000","last_peered":"2024-09-15T01:33:19.080783+0000","last_clean":"2024-09-15T01:33:19.080783+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:33:19.080783+0000","last_undegraded":"2024-09-15T01:33:19.080783+0000","last_fullsized":"2024-09-15T01:33:19.080783+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:39.303667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"78'71","reported_seq":351,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T01:33:20.122068+0000","last_change":"2024-09-15T01:33:20.123125+0000","last_active":"2024-09-15T01:33:20.122068+0000","last_peered":"2024-09-15T01:33:20.122068+0000","last_clean":"2024-09-15T01:33:20.122068+0000","last_became_active":"2024-09-15T01:33:12.081147+0000","last_became_peered":"2024-09-15T01:33:12.081147+0000","last_unstale":"2024-09-15T01:33:20.122068+0000","last_undegraded":"2024-09-15T01:33:20.122068+0000","last_fullsized":"2024-09-15T01:33:20.122068+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:52.191150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074669109999999997,"stat_sum":{"num_bytes":11173888,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":32,"num_read_kb":23291,"num_write":49,"num_write_kb":11072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":17530880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"},{"start":"27","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":64,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T01:33:19.697285+0000","last_change":"2024-09-15T01:32:25.568523+0000","last_active":"2024-09-15T01:33:19.697285+0000","last_peered":"2024-09-15T01:33:19.697285+0000","last_clean":"2024-09-15T01:33:19.697285+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:33:19.697285+0000","last_undegraded":"2024-09-15T01:33:19.697285+0000","last_fullsized":"2024-09-15T01:33:19.697285+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:33.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"76'50","reported_seq":274,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T01:33:20.118421+0000","last_change":"2024-09-15T01:33:20.118792+0000","last_active":"2024-09-15T01:33:20.118421+0000","last_peered":"2024-09-15T01:33:20.118421+0000","last_clean":"2024-09-15T01:33:20.118421+0000","last_became_active":"2024-09-15T01:33:12.081225+0000","last_became_peered":"2024-09-15T01:33:12.081225+0000","last_unstale":"2024-09-15T01:33:20.118421+0000","last_undegraded":"2024-09-15T01:33:20.118421+0000","last_fullsized":"2024-09-15T01:33:20.118421+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:53.748998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062441899999999997,"stat_sum":{"num_bytes":9715712,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":6812,"num_write":62,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":6586368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"2"},{"start":"21","length":"1"},{"start":"27","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":63,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T01:33:18.082967+0000","last_change":"2024-09-15T01:32:25.582726+0000","last_active":"2024-09-15T01:33:18.082967+0000","last_peered":"2024-09-15T01:33:18.082967+0000","last_clean":"2024-09-15T01:33:18.082967+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:33:18.082967+0000","last_undegraded":"2024-09-15T01:33:18.082967+0000","last_fullsized":"2024-09-15T01:33:18.082967+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:35.907962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":122044416,"num_objects":92,"num_object_clones":49,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":6,"num_read":324,"num_read_kb":95937,"num_write":731,"num_write_kb":152528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":44957696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":983,"ondisk_log_size":983,"up":112,"acting":112,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313234,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":824964,"kb_used_data":92855492,"kb_used_omap":2535,"kb_used_meta":0,"kb_avail":92854652,"statfs":{"total":95927926784,"available":95083163648,"internally_reserved":860160,"allocated":95084023808,"data_stored":95084023808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2596594,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.52800000000000002}]},{"osd":1,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.76600000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":30.641999999999999,"5min":30.641999999999999,"15min":30.641999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":574.26900000000001,"5min":574.26900000000001,"15min":574.26900000000001},"last":0.311},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40000000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:33:20 2024","interfaces":[{"interface":"back","average":{"1min":32.810000000000002,"5min":32.810000000000002,"15min":32.810000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":644.97699999999998,"5min":644.97699999999998,"15min":644.97699999999998},"last":644.97699999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.47199999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.0699999999999998,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.68700000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.747}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":7.3849999999999998,"5min":7.3849999999999998,"15min":7.3849999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":134.12200000000001,"5min":134.12200000000001,"15min":134.12200000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56899999999999995}]},{"osd":12,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.54300000000000004}]},{"osd":13,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.81399999999999995}]},{"osd":15,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.622}]}]},{"osd":9,"up_from":21,"seq":90194313231,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":811368,"kb_used_data":92869396,"kb_used_omap":861,"kb_used_meta":0,"kb_avail":92868248,"statfs":{"total":95927926784,"available":95097085952,"internally_reserved":1175552,"allocated":95098261504,"data_stored":95098261504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":882517,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.52600000000000002}]},{"osd":6,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.5709999999999997,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.442}]},{"osd":7,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.61199999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":10.916,"5min":10.916,"15min":10.916},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":13.147,"5min":13.147,"15min":13.147},"last":0.54700000000000004}]},{"osd":10,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":6.4539999999999997,"5min":6.4539999999999997,"15min":6.4539999999999997},"last":0.40400000000000003}]},{"osd":11,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9489999999999998,"5min":6.9489999999999998,"15min":6.9489999999999998},"last":0.57699999999999996}]},{"osd":12,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":29.687999999999999,"5min":29.687999999999999,"15min":29.687999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":495.95800000000003,"5min":495.95800000000003,"15min":495.95800000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":54.658000000000001,"5min":54.658000000000001,"15min":54.658000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":914.18499999999995,"5min":914.18499999999995,"15min":914.18499999999995},"last":0.58899999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.434},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":13.132999999999999,"5min":13.132999999999999,"15min":13.132999999999999},"last":0.51100000000000001}]},{"osd":14,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55700000000000005}]},{"osd":15,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.61899999999999999}]}]},{"osd":3,"up_from":21,"seq":90194313232,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":807640,"kb_used_data":92872548,"kb_used_omap":1082,"kb_used_meta":0,"kb_avail":92871976,"statfs":{"total":95927926784,"available":95100903424,"internally_reserved":585728,"allocated":95101489152,"data_stored":95101489152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1108508,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58899999999999997}]},{"osd":4,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4720000000000004,"5min":7.4720000000000004,"15min":7.4720000000000004},"last":0.442},{"interface":"front","average":{"1min":9.3040000000000003,"5min":9.3040000000000003,"15min":9.3040000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":182.87,"5min":182.87,"15min":182.87},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":6,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56999999999999995}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":24.288,"5min":24.288,"15min":24.288},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":500.678,"5min":500.678,"15min":500.678},"last":0.47299999999999998},{"interface":"front","average":{"1min":35.298999999999999,"5min":35.298999999999999,"15min":35.298999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":500.79899999999998,"5min":500.79899999999998,"15min":500.79899999999998},"last":0.54800000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":13,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":7.4500000000000002,"5min":7.4500000000000002,"15min":7.4500000000000002},"last":0.96699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]}]},{"osd":0,"up_from":21,"seq":90194313233,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":843408,"kb_used_data":92837548,"kb_used_omap":3517,"kb_used_meta":0,"kb_avail":92836208,"statfs":{"total":95927926784,"available":95064276992,"internally_reserved":1372160,"allocated":95065649152,"data_stored":95065649152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3601977,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.309},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":39.392000000000003,"5min":39.392000000000003,"15min":39.392000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":814.68299999999999,"5min":814.68299999999999,"15min":814.68299999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.49299999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.54600000000000004}]},{"osd":7,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.63700000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.59899999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":15,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.74099999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313232,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":822324,"kb_used_data":92857992,"kb_used_omap":2884,"kb_used_meta":0,"kb_avail":92857292,"statfs":{"total":95927926784,"available":95085867008,"internally_reserved":716800,"allocated":95086583808,"data_stored":95086583808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2953477,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.33800000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.1129999999999995,"5min":8.1129999999999995,"15min":8.1129999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":8.1430000000000007,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":0.67300000000000004}]},{"osd":3,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":19.710000000000001,"5min":19.710000000000001,"15min":19.710000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":342.56099999999998,"5min":342.56099999999998,"15min":342.56099999999998},"last":1.5069999999999999},{"interface":"front","average":{"1min":28.227,"5min":28.227,"15min":28.227},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":495.40800000000002,"5min":495.40800000000002,"15min":495.40800000000002},"last":1.4430000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.7960000000000003,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.7619999999999996,"5min":5.7619999999999996,"15min":5.7619999999999996},"last":1.2589999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.4779999999999998,"5min":6.4779999999999998,"15min":6.4779999999999998},"last":6.4779999999999998},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":5.726,"5min":5.726,"15min":5.726},"last":0.61499999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":44.985999999999997,"5min":44.985999999999997,"15min":44.985999999999997},"last":0.47999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":12,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53400000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.63}]},{"osd":15,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.78600000000000003}]}]},{"osd":2,"up_from":21,"seq":90194313231,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":826308,"kb_used_data":92854188,"kb_used_omap":2536,"kb_used_meta":0,"kb_avail":92853308,"statfs":{"total":95927926784,"available":95081787392,"internally_reserved":901120,"allocated":95082688512,"data_stored":95082688512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2597080,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":3,"commit_latency_ns":0,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.75700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.33800000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":6.6680000000000001,"5min":6.6680000000000001,"15min":6.6680000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":110.032,"5min":110.032,"15min":110.032},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.51500000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.495}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":11,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.69499999999999995}]},{"osd":12,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.60499999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.74299999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":15,"up_from":21,"seq":90194313232,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":819888,"kb_used_data":92860492,"kb_used_omap":1976,"kb_used_meta":0,"kb_avail":92859728,"statfs":{"total":95927926784,"available":95088361472,"internally_reserved":782336,"allocated":95089143808,"data_stored":95089143808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2023615,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.54400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.91000000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.52300000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.55600000000000005}]},{"osd":5,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.97099999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":5.8620000000000001,"5min":5.8620000000000001,"15min":5.8620000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":90.775999999999996,"5min":90.775999999999996,"15min":90.775999999999996},"last":0.624},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.92300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":34.816000000000003,"5min":34.816000000000003,"15min":34.816000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":542.06600000000003,"5min":542.06600000000003,"15min":542.06600000000003},"last":0.99099999999999999}]},{"osd":13,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.42999999999999999}]},{"osd":14,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.76600000000000001},{"interface":"front","average":{"1min":15.124000000000001,"5min":15.124000000000001,"15min":15.124000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":246.56100000000001,"5min":246.56100000000001,"15min":246.56100000000001},"last":1.224}]}]},{"osd":4,"up_from":21,"seq":90194313232,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":821240,"kb_used_data":92859724,"kb_used_omap":1583,"kb_used_meta":0,"kb_avail":92858376,"statfs":{"total":95927926784,"available":95086977024,"internally_reserved":1380352,"allocated":95088357376,"data_stored":95088357376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1621871,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.76000000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.68600000000000005},{"interface":"front","average":{"1min":19.617999999999999,"5min":19.617999999999999,"15min":19.617999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":361.452,"5min":361.452,"15min":361.452},"last":0.59999999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.63400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69799999999999995}]},{"osd":9,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.626},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.71499999999999997}]},{"osd":10,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.747}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.83199999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.80000000000000004}]}]},{"osd":5,"up_from":21,"seq":90194313231,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":834904,"kb_used_data":92848488,"kb_used_omap":1789,"kb_used_meta":0,"kb_avail":92844712,"statfs":{"total":95927926784,"available":95072985088,"internally_reserved":3866624,"allocated":95076851712,"data_stored":95076851712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1832461,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.57099999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.52000000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.433},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.58299999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.54100000000000004}]},{"osd":13,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.379}]},{"osd":14,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":24.149000000000001,"5min":24.149000000000001,"15min":24.149000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":492.697,"5min":492.697,"15min":492.697},"last":0.46999999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]}]},{"osd":6,"up_from":21,"seq":90194313231,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":833324,"kb_used_data":92849416,"kb_used_omap":1807,"kb_used_meta":0,"kb_avail":92846292,"statfs":{"total":95927926784,"available":95074603008,"internally_reserved":3198976,"allocated":95077801984,"data_stored":95077801984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1850451,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":4.3449999999999998,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70599999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.501}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.63900000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.52200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.77400000000000002}]},{"osd":10,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.71699999999999997}]},{"osd":11,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.60999999999999999}]},{"osd":12,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.627},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":5.8470000000000004,"5min":5.8470000000000004,"15min":5.8470000000000004},"last":0.74299999999999999}]},{"osd":13,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.73199999999999998}]}]},{"osd":7,"up_from":21,"seq":90194313232,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":814960,"kb_used_data":92864792,"kb_used_omap":1623,"kb_used_meta":0,"kb_avail":92864656,"statfs":{"total":95927926784,"available":95093407744,"internally_reserved":139264,"allocated":95093547008,"data_stored":95093547008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1662093,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":59.375,"5min":59.375,"15min":59.375},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":874.39499999999998,"5min":874.39499999999998,"15min":874.39499999999998},"last":0.78200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":6,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.878},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":5.6539999999999999,"5min":5.6539999999999999,"15min":5.6539999999999999},"last":0.69699999999999995}]},{"osd":8,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.86499999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":5.3860000000000001,"5min":5.3860000000000001,"15min":5.3860000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":62.991999999999997,"5min":62.991999999999997,"15min":62.991999999999997},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.72999999999999998},{"interface":"front","average":{"1min":54.116,"5min":54.116,"15min":54.116},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":792.73599999999999,"5min":792.73599999999999,"15min":792.73599999999999},"last":0.89800000000000002}]},{"osd":13,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":8.1489999999999991,"5min":8.1489999999999991,"15min":8.1489999999999991},"last":0.75700000000000001}]},{"osd":14,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.91700000000000004},{"interface":"front","average":{"1min":52.212000000000003,"5min":52.212000000000003,"15min":52.212000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":773.18799999999999,"5min":773.18799999999999,"15min":773.18799999999999},"last":0.82199999999999995}]},{"osd":15,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.66600000000000004}]}]},{"osd":8,"up_from":21,"seq":90194313231,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":825956,"kb_used_data":92854440,"kb_used_omap":2476,"kb_used_meta":0,"kb_avail":92853660,"statfs":{"total":95927926784,"available":95082147840,"internally_reserved":798720,"allocated":95082946560,"data_stored":95082946560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2536073,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":3,"commit_latency_ns":0,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.28399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.434},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.44700000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.59499999999999997}]},{"osd":9,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48499999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.53200000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52400000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.58499999999999996}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313231,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":816252,"kb_used_data":92864204,"kb_used_omap":2290,"kb_used_meta":0,"kb_avail":92863364,"statfs":{"total":95927926784,"available":95092084736,"internally_reserved":860160,"allocated":95092944896,"data_stored":95092944896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2345066,"internal_metadata":0},"hb_peers":[0,1,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.899},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40500000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.9500000000000002,"5min":3.9500000000000002,"15min":3.9500000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":4.4180000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":64.938999999999993,"5min":64.938999999999993,"15min":64.938999999999993},"last":0.49199999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.8959999999999999,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.51200000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":55.476999999999997,"5min":55.476999999999997,"15min":55.476999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":500.92099999999999,"5min":500.92099999999999,"15min":500.92099999999999},"last":0.439},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68000000000000005}]},{"osd":11,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.63800000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":4.2949999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.57299999999999995}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.4460000000000002},{"interface":"front","average":{"1min":36.435000000000002,"5min":36.435000000000002,"15min":36.435000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":641.649,"5min":641.649,"15min":641.649},"last":2.4740000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61899999999999999}]},{"osd":15,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.0670000000000002,"5min":6.0670000000000002,"15min":6.0670000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.69199999999999995}]}]},{"osd":11,"up_from":21,"seq":90194313231,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":820784,"kb_used_data":92859992,"kb_used_omap":1891,"kb_used_meta":0,"kb_avail":92858832,"statfs":{"total":95927926784,"available":95087443968,"internally_reserved":1187840,"allocated":95088631808,"data_stored":95088631808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1936635,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":46.469000000000001,"5min":46.469000000000001,"15min":46.469000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":873.95500000000004,"5min":873.95500000000004,"15min":873.95500000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.954,"5min":1.954,"15min":1.954},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":28.431999999999999,"5min":28.431999999999999,"15min":28.431999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":10,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":0.80900000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":5.5069999999999997,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":5.3120000000000003,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":0.65000000000000002}]},{"osd":13,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":25.056999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.70099999999999996}]},{"osd":14,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":16.126000000000001,"5min":16.126000000000001,"15min":16.126000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":310.78199999999998,"5min":310.78199999999998,"15min":310.78199999999998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":0.89600000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":12,"up_from":21,"seq":90194313232,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":809740,"kb_used_data":92869992,"kb_used_omap":1090,"kb_used_meta":0,"kb_avail":92869876,"statfs":{"total":95927926784,"available":95098753024,"internally_reserved":118784,"allocated":95098871808,"data_stored":95098871808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1116327,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.38100000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":8.0050000000000008,"5min":8.0050000000000008,"15min":8.0050000000000008},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":0.44900000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.46700000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.436},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.55700000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":24.937000000000001,"5min":24.937000000000001,"15min":24.937000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":488.69,"5min":488.69,"15min":488.69},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5419999999999998,"5min":3.5419999999999998,"15min":3.5419999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.6309999999999998,"5min":3.6309999999999998,"15min":3.6309999999999998},"last":0.755},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.76800000000000002}]},{"osd":11,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.70099999999999996}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.68400000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":15,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.81299999999999994}]}]},{"osd":1,"up_from":21,"seq":90194313232,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":824544,"kb_used_data":92856564,"kb_used_omap":2255,"kb_used_meta":0,"kb_avail":92855072,"statfs":{"total":95927926784,"available":95083593728,"internally_reserved":1527808,"allocated":95085121536,"data_stored":95085121536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2309305,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.66000000000000003}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.44600000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56499999999999995}]},{"osd":4,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":35.843000000000004,"5min":35.843000000000004,"15min":35.843000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":635.46400000000006,"5min":635.46400000000006,"15min":635.46400000000006},"last":0.58799999999999997}]},{"osd":5,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.31},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":6.4690000000000003,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.33900000000000002}]},{"osd":6,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.751}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.70799999999999996}]},{"osd":8,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.503}]},{"osd":10,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":27.920000000000002,"5min":27.920000000000002,"15min":27.920000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":9.9450000000000003,"5min":9.9450000000000003,"15min":9.9450000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":166.16200000000001,"5min":166.16200000000001,"15min":166.16200000000001},"last":0.77700000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.60399999999999998}]}]}],"pool_statfs":[]}} 2024-09-15T01:33:21.103 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [10, 8] 2024-09-15T01:33:21.104 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.0', '10', '8'] 2024-09-15T01:33:21.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.0 10 8 2024-09-15T01:33:21.105 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:21.103+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:21.206 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:21.204+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:21.228 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:21.228 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:21.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:33:21.307 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:21.306+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:21.373 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:21.374 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:21.382 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:33:21.400 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:33:21.443 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:21.442+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:21.478 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:21.478 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:21.486 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:33:21.529 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:21.529 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:21.529 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:21.528+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:21.549 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:21.549 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:21.570 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:21.570 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:21.583 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:21.583 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:21.589 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:21.589 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:21.592 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:33:21.607 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:21.608 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:21.628 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:21.628 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:21.633 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:21.631+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:21.648 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:21.648 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:21.669 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:21.669 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:21.687 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:21.687 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:21.707 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:21.707 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:21.709 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:21.706+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:21.726 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:21.726 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:21.729 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:21.729 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:21.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-15T01:33:21.745 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:21.745 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:21.764 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:21.764 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:21.784 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:21.784 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:21.803 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:21.803 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:21.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:21.802+0000 7f9c62d9d640 0 -- 172.21.15.42:0/4253997252 >> [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] conn(0x7f9c64195ff0 msgr2=0x7f9c641983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:21.810 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:21.808+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:21.835 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:21.836 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:21.844 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-15T01:33:21.911 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:21.909+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:21.940 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:21.940 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:21.949 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-15T01:33:22.012 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:22.010+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:22.023 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:22.048 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:22.048 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:11 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 45.868672033000003, 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032033109999999997, 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 11 2024-09-15T01:33:22.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176990+0000", 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2200000000000002e-07 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176995+0000", 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8040000000000001e-06 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177006+0000", 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1276e-05 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177012+0000", 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9370000000000001e-06 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177055+0000", 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2472999999999998e-05 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177197+0000", 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142519 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177567+0000", 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036945000000000001 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177659+0000", 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1928999999999993e-05 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177694+0000", 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4885000000000002e-05 2024-09-15T01:33:22.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177722+0000", 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8334e-05 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177733+0000", 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1491000000000001e-05 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208963+0000", 2024-09-15T01:33:22.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031230106000000001 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209022+0000", 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8984000000000003e-05 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:12 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:22.055 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 45.867921764999998, 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033440417, 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 12 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:22.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:22.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2500000000000005e-07 2024-09-15T01:33:22.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177745+0000", 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7269999999999998e-06 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177755+0000", 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9000000000000001e-06 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177760+0000", 2024-09-15T01:33:22.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1150000000000002e-06 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177801+0000", 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0802e-05 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177877+0000", 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6604e-05 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209067+0000", 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031189505999999999 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209144+0000", 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6728000000000001e-05 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211180+0000", 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020363099999999999 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.060 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=12, reqid=client.4477.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 45.855717552000002, 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022201101000000001, 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:22.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.75e-07 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189957+0000", 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2531000000000001e-05 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211941+0000", 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021983546999999999 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211947+0000", 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7329999999999997e-06 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211977+0000", 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9607e-05 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211982+0000", 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6290000000000001e-06 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212053+0000", 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1890000000000005e-05 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212060+0000", 2024-09-15T01:33:22.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3339999999999998e-06 2024-09-15T01:33:22.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212087+0000", 2024-09-15T01:33:22.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7314e-05 2024-09-15T01:33:22.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212098+0000", 2024-09-15T01:33:22.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1141999999999999e-05 2024-09-15T01:33:22.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212145+0000", 2024-09-15T01:33:22.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699000000000001e-05 2024-09-15T01:33:22.068 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.068 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:72 3.5s0 3:af8bec4d:::smithi17529310-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 45.838344818000003, 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.037268951000000002, 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.069 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.34e-07 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207319+0000", 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207326+0000", 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4860000000000002e-06 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207330+0000", 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0690000000000003e-06 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208687+0000", 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001356966 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208760+0000", 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2663000000000005e-05 2024-09-15T01:33:22.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210380+0000", 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001620659 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210464+0000", 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3387999999999995e-05 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.244586+0000", 2024-09-15T01:33:22.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034121925999999997 2024-09-15T01:33:22.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.074 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.074 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.0s2 34/31 ECSubWrite(tid=15, reqid=client.4477.0:74, at_version=34'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:33:22.074 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:22.074 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 45.833234071, 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069361051000000007, 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212428+0000", 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3399999999999999e-07 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212444+0000", 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5681000000000001e-05 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281357+0000", 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068913881999999996 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281366+0000", 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9740000000000008e-06 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281415+0000", 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8498000000000002e-05 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281421+0000", 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4590000000000003e-06 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281487+0000", 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5590000000000001e-05 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281500+0000", 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3298e-05 2024-09-15T01:33:22.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281575+0000", 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5247000000000006e-05 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281597+0000", 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1872999999999999e-05 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281788+0000", 2024-09-15T01:33:22.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000191015 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 38/31 ECSubWrite(tid=17, reqid=client.4477.0:208, at_version=38'12, trim_to=0'0, roll_forward_to=36'8))", 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.824300172999997, 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.93429882900000005, 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221362+0000", 2024-09-15T01:33:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9699999999999996e-07 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221380+0000", 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.804e-05 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.154929+0000", 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.93354887600000003 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.154938+0000", 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6220000000000004e-06 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155018+0000", 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0136000000000007e-05 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155027+0000", 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8259999999999999e-06 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155116+0000", 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8939999999999999e-05 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155127+0000", 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0981e-05 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155185+0000", 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7828999999999998e-05 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155204+0000", 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8649999999999999e-05 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155660+0000", 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000456432 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s3 44/38 ECSubWrite(tid=22, reqid=client.4477.0:293, at_version=44'14, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.271270+0000", 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.774391711, 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015434979, 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271270+0000", 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271270+0000", 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271270+0000", 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1999999999999999e-07 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271482+0000", 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021116799999999999 2024-09-15T01:33:22.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271510+0000", 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8881999999999999e-05 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271517+0000", 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3609999999999997e-06 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271571+0000", 2024-09-15T01:33:22.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4193000000000001e-05 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271577+0000", 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3849999999999999e-06 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271936+0000", 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035823400000000003 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.271967+0000", 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1644999999999999e-05 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.272044+0000", 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6448999999999999e-05 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.272061+0000", 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7524999999999999e-05 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.286705+0000", 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014643417000000001 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 51/31 ECSubWrite(tid=56, reqid=osd.5.0:56, at_version=51'43, trim_to=0'0, roll_forward_to=50'41))", 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.968810438999999, 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.063539848999999995, 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.091 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076876+0000", 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4834999999999999e-05 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139053+0000", 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.062177018000000001 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139067+0000", 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3842999999999999e-05 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139206+0000", 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013896500000000001 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139221+0000", 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5193e-05 2024-09-15T01:33:22.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139472+0000", 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025116 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139504+0000", 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1282999999999998e-05 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139541+0000", 2024-09-15T01:33:22.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7518999999999998e-05 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139557+0000", 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5781e-05 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.140391+0000", 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00083395699999999997 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.096 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:505 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [copy-from ver 2] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 24.972623096, 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15814687399999999, 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 505 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.097 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073040+0000", 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.792e-06 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073047+0000", 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0169999999999999e-06 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073064+0000", 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.668e-05 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073081+0000", 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7000999999999999e-05 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073158+0000", 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6779000000000005e-05 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073284+0000", 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000126802 2024-09-15T01:33:22.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082517+0000", 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0092324069999999998 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082822+0000", 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030544099999999999 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082921+0000", 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8760999999999999e-05 2024-09-15T01:33:22.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082994+0000", 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2769999999999996e-05 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083217+0000", 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000223706 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231110+0000", 2024-09-15T01:33:22.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.147892096 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231185+0000", 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5622000000000002e-05 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 55/31 ECSubWriteReply(tid=49, last_complete=55'33, committed=1, applied=1))", 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 24.961379783000002, 2024-09-15T01:33:22.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14718672399999999, 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9599999999999998e-07 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084287+0000", 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.230928+0000", 2024-09-15T01:33:22.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14664121799999999 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.230933+0000", 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4089999999999996e-06 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231018+0000", 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5589999999999999e-05 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231468+0000", 2024-09-15T01:33:22.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045001999999999998 2024-09-15T01:33:22.107 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.107 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.107 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.107 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s4 56/54 ECSubWrite(tid=228, reqid=client.4477.0:520, at_version=56'33, trim_to=0'0, roll_forward_to=55'31))", 2024-09-15T01:33:22.107 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:22.107 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.971659752000001, 2024-09-15T01:33:22.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016830581000000001, 2024-09-15T01:33:22.107 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7000000000000003e-07 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074027+0000", 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4845e-05 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074072+0000", 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4583000000000003e-05 2024-09-15T01:33:22.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074080+0000", 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6699999999999994e-06 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074280+0000", 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020015699999999999 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074290+0000", 2024-09-15T01:33:22.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0586e-05 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074562+0000", 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027202599999999997 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074573+0000", 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0974e-05 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074624+0000", 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0385e-05 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074786+0000", 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016278399999999999 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.090832+0000", 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016045700999999999 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.112 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 66/31 ECSubWrite(tid=130, reqid=osd.14.0:130, at_version=66'56, trim_to=0'0, roll_forward_to=60'52))", 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.965402092, 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.31166652299999997, 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2000000000000001e-07 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080275+0000", 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.556e-05 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.390960+0000", 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.31068479999999998 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.390970+0000", 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0223e-05 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391016+0000", 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5972e-05 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391025+0000", 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8610000000000004e-06 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391232+0000", 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020709799999999999 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391264+0000", 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2347e-05 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391319+0000", 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5096999999999998e-05 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391336+0000", 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6212e-05 2024-09-15T01:33:22.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391926+0000", 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00059013299999999996 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:646 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [copy-from ver 59] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.077480+0000", 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.9681817979999998, 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024635151000000001, 2024-09-15T01:33:22.118 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 646 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077480+0000", 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077480+0000", 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077482+0000", 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.967e-06 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077494+0000", 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2351000000000001e-05 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077510+0000", 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6461e-05 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077528+0000", 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7297000000000001e-05 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077583+0000", 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4877999999999999e-05 2024-09-15T01:33:22.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op must wait for map", 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077588+0000", 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5809999999999996e-06 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.078844+0000", 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001255693 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.078981+0000", 2024-09-15T01:33:22.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013696499999999999 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097046+0000", 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018065193 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097686+0000", 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00063973600000000002 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097704+0000", 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7934000000000001e-05 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098011+0000", 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030703799999999998 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098161+0000", 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000150233 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.102063+0000", 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003902109 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.102115+0000", 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1715000000000002e-05 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.125 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:647 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.077541+0000", 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.9681206509999996, 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020616064, 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 647 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077541+0000", 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077541+0000", 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077542+0000", 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2099999999999995e-07 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077545+0000", 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5360000000000001e-06 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077555+0000", 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0193e-05 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077561+0000", 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5679999999999999e-06 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077630+0000", 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9648999999999997e-05 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077635+0000", 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0050000000000004e-06 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.079228+0000", 2024-09-15T01:33:22.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015930009999999999 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.079271+0000", 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2588e-05 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097903+0000", 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018632124999999999 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098048+0000", 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000144979 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098157+0000", 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010859899999999999 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:676 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [setxattr _header (56) in=7b] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:22.131 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.7750908679999995, 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054773589999999997, 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 676 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1199999999999999e-07 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270573+0000", 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7409999999999999e-06 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270576+0000", 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0580000000000002e-06 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270579+0000", 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0630000000000002e-06 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273029+0000", 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024502119999999998 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273139+0000", 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110196 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273456+0000", 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031676000000000002 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273543+0000", 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7276999999999998e-05 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273550+0000", 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2119999999999997e-06 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273583+0000", 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2625e-05 2024-09-15T01:33:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273919+0000", 2024-09-15T01:33:22.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033556900000000003 2024-09-15T01:33:22.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:22.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325294+0000", 2024-09-15T01:33:22.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051375250999999997 2024-09-15T01:33:22.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325344+0000", 2024-09-15T01:33:22.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0314e-05 2024-09-15T01:33:22.138 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.138 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.138 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.138 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:677 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:22.138 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.7750695739999998, 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.058358134999999998, 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 677 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:22.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.65e-07 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270594+0000", 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4219999999999999e-06 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270598+0000", 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9439999999999998e-06 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270600+0000", 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.243e-06 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.274286+0000", 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036864089999999999 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.274337+0000", 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0677999999999998e-05 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325802+0000", 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051465576999999998 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325904+0000", 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010192 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328950+0000", 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030455769999999998 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.143 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 71/31 ECSubWriteReply(tid=63, last_complete=71'46, committed=1, applied=1))", 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.273846+0000", 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.7718150240000003, 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051825606000000003, 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273846+0000", 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273846+0000", 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273847+0000", 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5199999999999997e-07 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273850+0000", 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1690000000000001e-06 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325190+0000", 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051340157999999997 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325198+0000", 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4839999999999998e-06 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325235+0000", 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7156000000000001e-05 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325672+0000", 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00043708700000000001 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 71/31 ECSubWriteReply(tid=64, last_complete=71'46, committed=1, applied=1))", 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.329874+0000", 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.7157873939999995, 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.074638840999999997, 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.147 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329874+0000", 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329874+0000", 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329875+0000", 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2799999999999996e-07 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329878+0000", 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7579999999999998e-06 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404471+0000", 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.074592695000000001 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404475+0000", 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8650000000000003e-06 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404497+0000", 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2458000000000001e-05 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404513+0000", 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5537e-05 2024-09-15T01:33:22.150 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:714 3.5s0 3:a0a9f8ce:::smithi17529310-35:head [delete] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+full_force+supports_pool_eio e71)", 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.401661+0000", 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.6440005430000006, 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044420634000000001, 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.151 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 714 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401661+0000", 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401661+0000", 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401663+0000", 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401670+0000", 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3549999999999999e-06 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401687+0000", 2024-09-15T01:33:22.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7388e-05 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401695+0000", 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6310000000000003e-06 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401732+0000", 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7415e-05 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401813+0000", 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0117999999999994e-05 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402335+0000", 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052289800000000005 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402584+0000", 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024863700000000001 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402594+0000", 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7890000000000006e-06 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402634+0000", 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9984000000000002e-05 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402647+0000", 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3105000000000001e-05 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446020+0000", 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.043373215 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446081+0000", 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1248999999999996e-05 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 71/31 ECSubWriteReply(tid=66, last_complete=71'48, committed=1, applied=1))", 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.6412066870000004, 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041833197000000003, 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2099999999999997e-07 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404458+0000", 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.976e-06 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.445921+0000", 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041463284000000003 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.445925+0000", 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0260000000000001e-06 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.445975+0000", 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9424e-05 2024-09-15T01:33:22.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:22.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:22.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:22.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446288+0000", 2024-09-15T01:33:22.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031296600000000003 2024-09-15T01:33:22.161 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.161 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.161 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.161 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:22.161 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:22.161 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:22.162 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-15T01:33:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:set 2.0 pg_upmap mapping to [10,8] 2024-09-15T01:33:22.164 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:33:22.165 INFO:tasks.rados.rados.0.smithi175.stdout:307: done (0 left) 2024-09-15T01:33:22.165 INFO:tasks.rados.rados.0.smithi175.stdout:308: rollback oid 31 current snap is 26 2024-09-15T01:33:22.165 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 31 to 23 2024-09-15T01:33:22.165 INFO:tasks.rados.rados.0.smithi175.stdout:309: snap_remove snap 20 2024-09-15T01:33:22.166 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:22.160+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:22.263 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:22.262+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:22.306 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:22.306 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:22.306 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:33:22.306 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:22.315 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:33:22.399 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:22.397+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:22.456 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:22.456 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:22.456 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:33:22.456 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:33:22.456 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:22.464 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:33:22.506 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:22.504+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:22.566 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:22.565+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:22.607 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:33:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:33:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:33:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.7s3 34/31 ECSubWrite(tid=17, reqid=client.4477.0:77, at_version=34'6, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:33:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:33:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.389125409000002, 2024-09-15T01:33:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015250150000000001, 2024-09-15T01:33:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215518+0000", 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3567000000000001e-05 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230278+0000", 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014760300000000001 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230285+0000", 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.285e-06 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230359+0000", 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4025999999999999e-05 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230367+0000", 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4500000000000004e-06 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230457+0000", 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0457999999999993e-05 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230498+0000", 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0204000000000003e-05 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230554+0000", 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6097999999999997e-05 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230573+0000", 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9252000000000001e-05 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230754+0000", 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018093300000000001 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:90 3.6s0 3:65d6c4b7:::smithi17529310-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.378487491000001, 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.964412163, 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 90 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226144+0000", 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0490000000000002e-06 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226152+0000", 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5700000000000004e-06 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226157+0000", 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0069999999999997e-06 2024-09-15T01:33:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.227618+0000", 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014613 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.227698+0000", 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9518000000000007e-05 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.228785+0000", 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010872729999999999 2024-09-15T01:33:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.228833+0000", 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.85e-05 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:37.179967+0000", 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.95113379099999995 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:37.179971+0000", 2024-09-15T01:33:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7620000000000001e-06 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:38.180851+0000", 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0008801570000001 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:38.180855+0000", 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.416e-06 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.187397+0000", 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.006542346 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.187673+0000", 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000275825 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.190554+0000", 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028811650000000002 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.7s3 34/31 ECSubRead(tid=22, to_read={3:e6e8875f:::smithi17529310-31:head=0,4096,0}, subchunks={3:e6e8875f:::smithi17529310-31:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.371115445999997, 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.074844447999999994, 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4300000000000003e-07 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233521+0000", 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0280000000000003e-06 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308171+0000", 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.074650066000000001 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308178+0000", 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5810000000000002e-06 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308258+0000", 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9977999999999999e-05 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308358+0000", 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100152 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:196 3.6s0 3:65d6c4b7:::smithi17529310-30:head [append 0~0] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.401467009000001, 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007181882, 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 196 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203168+0000", 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5620000000000003e-06 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203175+0000", 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5599999999999996e-06 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203194+0000", 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8757999999999999e-05 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203201+0000", 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7429999999999996e-06 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203414+0000", 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021300600000000001 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203419+0000", 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1420000000000001e-06 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.204700+0000", 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.001280647 2024-09-15T01:33:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.204945+0000", 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024550199999999998 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.205894+0000", 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094873799999999997 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206188+0000", 2024-09-15T01:33:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029375700000000001 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206257+0000", 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9632e-05 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206371+0000", 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011411100000000001 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206444+0000", 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2355999999999996e-05 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210281+0000", 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038376740000000001 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210344+0000", 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2694000000000007e-05 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:197 3.6s0 3:65d6c4b7:::smithi17529310-30:head [rollback 3] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.401413415, 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0090962939999999, 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 197 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8299999999999997e-07 2024-09-15T01:33:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203220+0000", 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0620000000000002e-06 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203229+0000", 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5329999999999994e-06 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203234+0000", 2024-09-15T01:33:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000002e-06 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203537+0000", 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030286700000000001 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203539+0000", 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.277e-06 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206748+0000", 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.003209102 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206862+0000", 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011395000000000001 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207832+0000", 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097048299999999996 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207937+0000", 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104583 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207956+0000", 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9381e-05 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208001+0000", 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4642999999999999e-05 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209901+0000", 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001899766 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211474+0000", 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015733190000000001 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212312+0000", 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083790500000000001 2024-09-15T01:33:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:198 3.6s0 3:65d6c4b7:::smithi17529310-30:head [append 0~0] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.401129222999998, 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.009021902, 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 198 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203501+0000", 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4600000000000004e-07 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203505+0000", 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3830000000000002e-06 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203562+0000", 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7120000000000002e-05 2024-09-15T01:33:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203570+0000", 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203656+0000", 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6273000000000006e-05 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203658+0000", 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4360000000000001e-06 2024-09-15T01:33:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208296+0000", 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004637574 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208407+0000", 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110994 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209048+0000", 2024-09-15T01:33:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064081900000000005 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209141+0000", 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3827999999999997e-05 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209164+0000", 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2201e-05 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209206+0000", 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2021999999999998e-05 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209910+0000", 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070434399999999998 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211939+0000", 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020290159999999998 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212522+0000", 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058283599999999999 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:211 3.6s0 3:6ba0b44a:::smithi17529310-6:head [copy-from ver 0] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.406648695999998, 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.22039597699999999, 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197982+0000", 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.618e-06 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197988+0000", 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4519999999999996e-06 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198027+0000", 2024-09-15T01:33:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9492000000000001e-05 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198036+0000", 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3950000000000007e-06 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198736+0000", 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00069949200000000002 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198740+0000", 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3470000000000001e-06 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210425+0000", 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011684905000000001 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210622+0000", 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019688099999999999 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418376+0000", 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.20775439500000001 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:212 3.6s0 3:6ba0b44a:::smithi17529310-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.406585047, 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.22052332099999999, 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 212 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:33:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198045+0000", 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6499999999999998e-07 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198049+0000", 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4150000000000001e-06 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198058+0000", 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9830000000000002e-06 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198064+0000", 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6219999999999997e-06 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198782+0000", 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071748199999999997 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198783+0000", 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9400000000000001e-06 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210808+0000", 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012024217 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210861+0000", 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2859999999999999e-05 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418443+0000", 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.20758253099999999 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418509+0000", 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5516000000000001e-05 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418567+0000", 2024-09-15T01:33:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.889e-05 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs6 39/31 ECSubWrite(tid=56, reqid=osd.11.0:56, at_version=39'21, trim_to=0'0, roll_forward_to=38'17))", 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.361836295000003, 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038235071000000002, 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242794+0000", 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8599999999999997e-07 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242817+0000", 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3655e-05 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280383+0000", 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037566100999999998 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280390+0000", 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:33:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280442+0000", 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1650000000000002e-05 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280448+0000", 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7479999999999999e-06 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280660+0000", 2024-09-15T01:33:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021205900000000001 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280667+0000", 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4710000000000001e-06 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280705+0000", 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7511000000000001e-05 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280722+0000", 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7646999999999999e-05 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281028+0000", 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030575099999999999 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs2 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:33:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.362314222999998, 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011429094000000001, 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242316+0000", 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2900000000000003e-07 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242337+0000", 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1263999999999998e-05 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242392+0000", 2024-09-15T01:33:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5408999999999998e-05 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242399+0000", 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.866e-06 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242456+0000", 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-05 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242466+0000", 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0015e-05 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242685+0000", 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021821399999999999 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242713+0000", 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8589e-05 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242754+0000", 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1173e-05 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242790+0000", 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5167999999999999e-05 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253744+0000", 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010954527 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs1 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.361817838999997, 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010913366000000001, 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242812+0000", 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242834+0000", 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1827999999999998e-05 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242901+0000", 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6840000000000004e-05 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242908+0000", 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5440000000000001e-06 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242959+0000", 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1057000000000003e-05 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242969+0000", 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8030000000000008e-06 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243189+0000", 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021987499999999999 2024-09-15T01:33:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243212+0000", 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2569999999999999e-05 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243258+0000", 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6452000000000002e-05 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243274+0000", 2024-09-15T01:33:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6426000000000002e-05 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253725+0000", 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010450249 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs6 44/31 ECSubWrite(tid=82, reqid=client.4477.0:281, at_version=44'36, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.361064757000001, 2024-09-15T01:33:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011937237, 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243565+0000", 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0500000000000003e-07 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243585+0000", 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0313e-05 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243689+0000", 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103977 2024-09-15T01:33:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243696+0000", 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0489999999999998e-06 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243791+0000", 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4394000000000005e-05 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243800+0000", 2024-09-15T01:33:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2439999999999999e-06 2024-09-15T01:33:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244285+0000", 2024-09-15T01:33:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000484802 2024-09-15T01:33:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244313+0000", 2024-09-15T01:33:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8515000000000001e-05 2024-09-15T01:33:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244359+0000", 2024-09-15T01:33:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5389000000000002e-05 2024-09-15T01:33:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244375+0000", 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6699000000000001e-05 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.255502+0000", 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011126250000000001 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 44/31 ECSubWrite(tid=39, reqid=client.4477.0:283, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.336342766999998, 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012672954, 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268287+0000", 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.85e-07 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268806+0000", 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00051859699999999998 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272018+0000", 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032124620000000001 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272032+0000", 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4008e-05 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272076+0000", 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3807999999999998e-05 2024-09-15T01:33:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272085+0000", 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9749999999999996e-06 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272331+0000", 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024564600000000003 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272349+0000", 2024-09-15T01:33:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7988000000000001e-05 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272412+0000", 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2718000000000005e-05 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272426+0000", 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.280959+0000", 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085329199999999994 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s1 55/31 ECSubWrite(tid=67, reqid=client.4477.0:503, at_version=0'0, trim_to=0'0, roll_forward_to=55'51))", 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.106011+0000", 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.498618355000001, 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017261852000000001, 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.106011+0000", 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.106011+0000", 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.106013+0000", 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2079999999999999e-06 2024-09-15T01:33:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.106037+0000", 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4114e-05 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122822+0000", 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016784701999999999 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122834+0000", 2024-09-15T01:33:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2055000000000001e-05 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122891+0000", 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7244999999999998e-05 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122899+0000", 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122994+0000", 2024-09-15T01:33:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4554000000000003e-05 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.123004+0000", 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.049e-05 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.123055+0000", 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0927999999999998e-05 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.123074+0000", 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9177999999999998e-05 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.123273+0000", 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019858899999999999 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 60/38 ECSubWriteReply(tid=51, last_complete=60'31, committed=1, applied=1))", 2024-09-15T01:33:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:02.100140+0000", 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.504488966, 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.402923223, 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.100140+0000", 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.100140+0000", 2024-09-15T01:33:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.100141+0000", 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9599999999999999e-07 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.100145+0000", 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1949999999999996e-06 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.502953+0000", 2024-09-15T01:33:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.40280827200000002 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.502961+0000", 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9840000000000001e-06 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.503036+0000", 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4394999999999995e-05 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.503063+0000", 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7781e-05 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:636 3.6s0 3:6ba0b44a:::smithi17529310-6:head [copy-from ver 36] snapc 21=[21,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:11.095326+0000", 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.509302799, 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0117660310000001, 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 636 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095326+0000", 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095326+0000", 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095328+0000", 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3859999999999999e-06 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095333+0000", 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0660000000000003e-06 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095345+0000", 2024-09-15T01:33:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1979e-05 2024-09-15T01:33:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095350+0000", 2024-09-15T01:33:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7400000000000004e-06 2024-09-15T01:33:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095405+0000", 2024-09-15T01:33:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5606999999999998e-05 2024-09-15T01:33:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095408+0000", 2024-09-15T01:33:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7520000000000001e-06 2024-09-15T01:33:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.081910+0000", 2024-09-15T01:33:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.98650184200000002 2024-09-15T01:33:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.082126+0000", 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021671099999999999 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102015+0000", 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019888449999999998 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102256+0000", 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024061300000000001 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102304+0000", 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8597000000000002e-05 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102756+0000", 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045214000000000002 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102920+0000", 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000163793 2024-09-15T01:33:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.107042+0000", 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041217019999999997 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.107092+0000", 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0652999999999998e-05 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:637 3.6s0 3:6ba0b44a:::smithi17529310-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:11.095356+0000", 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.509272707000001, 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0073326149999999, 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 637 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095356+0000", 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095356+0000", 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095357+0000", 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095358+0000", 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4130000000000001e-06 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095362+0000", 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0459999999999999e-06 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095365+0000", 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5409999999999999e-06 2024-09-15T01:33:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095430+0000", 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5026999999999999e-05 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095431+0000", 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0899999999999999e-06 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.084105+0000", 2024-09-15T01:33:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.98867371999999998 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.084176+0000", 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0876000000000004e-05 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102436+0000", 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018259958 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102560+0000", 2024-09-15T01:33:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012495599999999999 2024-09-15T01:33:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102689+0000", 2024-09-15T01:33:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012861900000000001 2024-09-15T01:33:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:701 3.6s0 3:6ba0b44a:::smithi17529310-6:head [copy-from ver 59] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.298859+0000", 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3057706010000008, 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.031040582000000001, 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 701 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298859+0000", 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298859+0000", 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298860+0000", 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8530000000000001e-06 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298868+0000", 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1969999999999999e-06 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298884+0000", 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6688999999999999e-05 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298891+0000", 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6290000000000004e-06 2024-09-15T01:33:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298945+0000", 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3533999999999999e-05 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.299722+0000", 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077747099999999996 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321205+0000", 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021483149999999999 2024-09-15T01:33:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321806+0000", 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060049599999999995 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321829+0000", 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3229000000000001e-05 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322419+0000", 2024-09-15T01:33:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058996099999999996 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322847+0000", 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042865900000000002 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.329826+0000", 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069782359999999996 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.329899+0000", 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3478000000000004e-05 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:702 3.6s0 3:6ba0b44a:::smithi17529310-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.298904+0000", 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3057255130000005, 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023906357, 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 702 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298904+0000", 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298904+0000", 2024-09-15T01:33:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298904+0000", 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6199999999999998e-07 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298907+0000", 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2179999999999999e-06 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298915+0000", 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.943e-06 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298919+0000", 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0160000000000002e-06 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.300003+0000", 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010835440000000001 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.300092+0000", 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8721000000000007e-05 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322365+0000", 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022273173 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322606+0000", 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024124699999999999 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322810+0000", 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000203933 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs6 76/31 ECSubWrite(tid=279, reqid=client.4477.0:751, at_version=76'153, trim_to=0'0, roll_forward_to=73'147))", 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:18.088335+0000", 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5162945289999996, 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01095546, 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088335+0000", 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088335+0000", 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088335+0000", 2024-09-15T01:33:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4399999999999999e-07 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088355+0000", 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0604000000000001e-05 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088849+0000", 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049329899999999999 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:22.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088855+0000", 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088918+0000", 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2182999999999997e-05 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088929+0000", 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1270000000000001e-05 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.089476+0000", 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00054738699999999998 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.089514+0000", 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7858999999999998e-05 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.089563+0000", 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8421999999999998e-05 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.089621+0000", 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8103999999999998e-05 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.099290+0000", 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0096694829999999996 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:22.726 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:22.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:33:22.728 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:22.726+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:22.828 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:22.827+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:22.870 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:22.870 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:22.870 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:33:22.870 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:22.878 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:33:22.929 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:22.927+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:22.978 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:33:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:33:22.979 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:22.987 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:33:23.030 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:23.029+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:23.081 INFO:tasks.rados.rados.0.smithi175.stdout:309: done (1 left) 2024-09-15T01:33:23.081 INFO:tasks.rados.rados.0.smithi175.stdout:310: snap_create 2024-09-15T01:33:23.081 INFO:tasks.rados.rados.0.smithi175.stdout:308: finishing rollback tid 0 to smithi17529310-31 2024-09-15T01:33:23.081 INFO:tasks.rados.rados.0.smithi175.stdout:308: finishing rollback tid 1 to smithi17529310-31 2024-09-15T01:33:23.081 INFO:tasks.rados.rados.0.smithi175.stdout:308: finishing rollback tid 2 to smithi17529310-31 2024-09-15T01:33:23.081 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 31 v 75 (ObjNum 85 snap 7 seq_num 85) dirty exists 2024-09-15T01:33:23.097 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:33:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:33:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:33:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:14 3.1s0 3:8930608f:::smithi17529310-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.176805+0000", 2024-09-15T01:33:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.91776728, 2024-09-15T01:33:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020710388, 2024-09-15T01:33:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 14 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176805+0000", 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176805+0000", 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176806+0000", 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176810+0000", 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8399999999999997e-06 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176824+0000", 2024-09-15T01:33:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4051e-05 2024-09-15T01:33:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176827+0000", 2024-09-15T01:33:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5159999999999999e-06 2024-09-15T01:33:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178153+0000", 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013253519999999999 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178241+0000", 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8670000000000003e-05 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178708+0000", 2024-09-15T01:33:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00046673299999999999 2024-09-15T01:33:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178860+0000", 2024-09-15T01:33:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015201099999999999 2024-09-15T01:33:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179098+0000", 2024-09-15T01:33:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238123 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179138+0000", 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0184e-05 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180215+0000", 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001076544 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197335+0000", 2024-09-15T01:33:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017120209000000001 2024-09-15T01:33:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197516+0000", 2024-09-15T01:33:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000180657 2024-09-15T01:33:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:15 3.1s0 3:8930608f:::smithi17529310-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.176858+0000", 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.917714193000002, 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023897231000000001, 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15 2024-09-15T01:33:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176858+0000", 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176858+0000", 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176859+0000", 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0999999999999999e-07 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176861+0000", 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4099999999999998e-06 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176875+0000", 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3366e-05 2024-09-15T01:33:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176882+0000", 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5560000000000002e-06 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178931+0000", 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002048299 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179014+0000", 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3207000000000005e-05 2024-09-15T01:33:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197559+0000", 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018545582000000001 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197656+0000", 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7085999999999998e-05 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.200756+0000", 2024-09-15T01:33:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030992150000000002 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:29 3.7s0 3:e2b4e7d9:::smithi17529310-10:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.177051+0000", 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.917521247000003, 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020495925000000002, 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 29 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177051+0000", 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177051+0000", 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177052+0000", 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177064+0000", 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1477999999999999e-05 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177072+0000", 2024-09-15T01:33:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5629999999999992e-06 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177084+0000", 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1557e-05 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178451+0000", 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013673019999999999 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178548+0000", 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6639999999999996e-05 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179029+0000", 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048111300000000003 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179171+0000", 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014193800000000001 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2196e-05 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179212+0000", 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9462999999999999e-05 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179251+0000", 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9155e-05 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197330+0000", 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018078409 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197547+0000", 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021747699999999999 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:30 3.7s0 3:e2b4e7d9:::smithi17529310-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.177099+0000", 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.917473508999997, 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024179603000000001, 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 30 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177099+0000", 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177099+0000", 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177100+0000", 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-07 2024-09-15T01:33:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177103+0000", 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.847e-06 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177111+0000", 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4780000000000008e-06 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177116+0000", 2024-09-15T01:33:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999998e-06 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179311+0000", 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002195331 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179405+0000", 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4611999999999995e-05 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197589+0000", 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018183300999999999 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197665+0000", 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5878999999999996e-05 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.201279+0000", 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003614089 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:31 3.1s0 3:8776790e:::smithi17529310-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.177756+0000", 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.916816873999998, 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020342740000000002, 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 31 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177756+0000", 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177756+0000", 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177757+0000", 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178059+0000", 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030214800000000002 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178069+0000", 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0324e-05 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178081+0000", 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1491999999999999e-05 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179041+0000", 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096040399999999999 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179333+0000", 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029163799999999999 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180034+0000", 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070128900000000001 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180155+0000", 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120844 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180166+0000", 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0967e-05 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180220+0000", 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4048999999999997e-05 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182483+0000", 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022628930000000002 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.198037+0000", 2024-09-15T01:33:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015553726 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.198099+0000", 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1824999999999998e-05 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:32 3.1s0 3:8776790e:::smithi17529310-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.178085+0000", 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.916487279000002, 2024-09-15T01:33:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021154846000000001, 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178085+0000", 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178085+0000", 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178086+0000", 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0699999999999997e-07 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178088+0000", 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1100000000000001e-06 2024-09-15T01:33:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178096+0000", 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4419999999999998e-06 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178101+0000", 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1810000000000003e-06 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180259+0000", 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021580839999999998 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180359+0000", 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100028 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181451+0000", 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001092569 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181638+0000", 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018707899999999999 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181720+0000", 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1682999999999997e-05 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182311+0000", 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059133299999999998 2024-09-15T01:33:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182487+0000", 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017578299999999999 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199166+0000", 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016678622000000001 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199240+0000", 2024-09-15T01:33:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4425000000000007e-05 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:33 3.1s0 3:8776790e:::smithi17529310-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.916467855, 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024100798, 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 33 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178108+0000", 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3259999999999998e-06 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178114+0000", 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5209999999999999e-06 2024-09-15T01:33:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178118+0000", 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7139999999999999e-06 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182355+0000", 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042371070000000004 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182467+0000", 2024-09-15T01:33:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011186799999999999 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199900+0000", 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017433015999999999 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199973+0000", 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2897000000000004e-05 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.202206+0000", 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022329049999999999 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:44 3.bs0 3:d6f740a8:::smithi17529310-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.185918+0000", 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.908654808999998, 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.123698503, 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185918+0000", 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185918+0000", 2024-09-15T01:33:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185919+0000", 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7400000000000003e-07 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185927+0000", 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7860000000000002e-06 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185936+0000", 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7569999999999993e-06 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185941+0000", 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1889999999999998e-06 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187237+0000", 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001295806 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187337+0000", 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9630000000000007e-05 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187837+0000", 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000500757 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187919+0000", 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1482000000000004e-05 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187962+0000", 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2647e-05 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188001+0000", 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9070999999999999e-05 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188078+0000", 2024-09-15T01:33:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7713999999999995e-05 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309538+0000", 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.121459186 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309616+0000", 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8804e-05 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:45 3.bs0 3:d6f740a8:::smithi17529310-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.908627035000002, 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12623172299999999, 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 45 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-15T01:33:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185948+0000", 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185954+0000", 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1090000000000003e-06 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185958+0000", 2024-09-15T01:33:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5159999999999999e-06 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187975+0000", 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002017343 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188054+0000", 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8734999999999997e-05 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309670+0000", 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.121615926 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309756+0000", 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6316999999999994e-05 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.312177+0000", 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002421126 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:78 3.7s0 3:eb933b0c:::smithi17529310-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.213590+0000", 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.880982461000002, 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020370672999999999, 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 78 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213590+0000", 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213590+0000", 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213591+0000", 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213593+0000", 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3199999999999998e-06 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213600+0000", 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4580000000000004e-06 2024-09-15T01:33:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213607+0000", 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-06 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215631+0000", 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020248890000000002 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215718+0000", 2024-09-15T01:33:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6127999999999999e-05 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231591+0000", 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015873015000000001 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231666+0000", 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5525999999999995e-05 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233961+0000", 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022948069999999998 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:93 3.7s0 3:e6e8875f:::smithi17529310-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.865187126999999, 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079343245000000007, 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 93 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1299999999999999e-07 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229390+0000", 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.703e-06 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229403+0000", 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3056e-05 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229410+0000", 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231135+0000", 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001725737 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231216+0000", 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0586000000000004e-05 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233251+0000", 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020346819999999999 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233339+0000", 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7907999999999999e-05 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308729+0000", 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.075390190999999995 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.7s0 47/38 ECSubWriteReply(tid=76, last_complete=47'25, committed=1, applied=1))", 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.730350117999997, 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.42564722900000002, 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8599999999999999e-07 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364224+0000", 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5099999999999999e-06 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789796+0000", 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.425571957 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789805+0000", 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789857+0000", 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1598999999999998e-05 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789870+0000", 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2850000000000001e-05 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpReadReply(3.1s0 53/52 ECSubReadReply(tid=94, attrs_read=0))", 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:55.205063+0000", 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.889509188000002, 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99844881399999996, 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:55.205063+0000", 2024-09-15T01:33:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:55.205063+0000", 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:55.205064+0000", 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:55.205072+0000", 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5299999999999996e-06 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.202784+0000", 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99771169100000001 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.202787+0000", 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.841e-06 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.202869+0000", 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1705999999999993e-05 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.203512+0000", 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064375100000000002 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpReadReply(3.7s0 54/52 ECSubReadReply(tid=102, attrs_read=0))", 2024-09-15T01:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.107553+0000", 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.987019433, 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.96435113900000002, 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.107553+0000", 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.107553+0000", 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.107554+0000", 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3500000000000002e-07 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.107690+0000", 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000136157 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.071644+0000", 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.96395441400000004 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.071651+0000", 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0419999999999997e-06 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.071704+0000", 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2580000000000001e-05 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.071904+0000", 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020051100000000001 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpReadReply(3.1s0 54/52 ECSubReadReply(tid=116, attrs_read=0))", 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.207373+0000", 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.887199193000001, 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.83479060699999996, 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.207373+0000", 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.207373+0000", 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.207374+0000", 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8599999999999998e-07 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.207523+0000", 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014929200000000001 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.041829+0000", 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.83430607300000004 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.041837+0000", 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9030000000000004e-06 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.041884+0000", 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6993999999999998e-05 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.042164+0000", 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027975899999999998 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:500 3.7s0 3:e89d09e8:::smithi17529310-19:head [setxattr _header (57) in=64b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.209208+0000", 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.885364449000001, 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.280345175, 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 500 2024-09-15T01:33:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209208+0000", 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209208+0000", 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209209+0000", 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4199999999999995e-07 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209213+0000", 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8410000000000002e-06 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209217+0000", 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.391e-06 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209220+0000", 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6570000000000001e-06 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.210612+0000", 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013926850000000001 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.210830+0000", 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000217655 2024-09-15T01:33:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211263+0000", 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043276600000000001 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211409+0000", 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014583400000000001 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211418+0000", 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7459999999999995e-06 2024-09-15T01:33:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211458+0000", 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999999e-05 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211653+0000", 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000195801 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.489314+0000", 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.27766080300000001 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.489553+0000", 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023910399999999999 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:501 3.7s0 3:e89d09e8:::smithi17529310-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:33:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.885350091999999, 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.28201526999999998, 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 501 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209225+0000", 2024-09-15T01:33:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9819999999999998e-06 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209229+0000", 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3959999999999999e-06 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209233+0000", 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5609999999999999e-06 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211492+0000", 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002259414 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211637+0000", 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014431099999999999 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.489907+0000", 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.27827002099999998 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.490070+0000", 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016353799999999999 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.491238+0000", 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011676309999999999 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.7s0 54/52 ECSubWriteReply(tid=119, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.211852+0000", 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.882720730999999, 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19905882499999999, 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211852+0000", 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211852+0000", 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211853+0000", 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8500000000000001e-07 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211858+0000", 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9049999999999996e-06 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.410806+0000", 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.198948825 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.410818+0000", 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1161e-05 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.410890+0000", 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2223000000000003e-05 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.410911+0000", 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1026000000000002e-05 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:748 3.7s0 3:e89d09e8:::smithi17529310-19:head [copy-from ver 67] snapc 27=[27,26,24,22,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:18.077994+0000", 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0165785249999999, 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.031699640000000001, 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 748 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.077994+0000", 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.077994+0000", 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.077996+0000", 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.339e-06 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078006+0000", 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9359999999999994e-06 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078024+0000", 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7912999999999999e-05 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078032+0000", 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9549999999999992e-06 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078096+0000", 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3443999999999998e-05 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078104+0000", 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0169999999999996e-06 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.079874+0000", 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001769851 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.079984+0000", 2024-09-15T01:33:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110844 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.103104+0000", 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023119681 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.103642+0000", 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000537906 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.103661+0000", 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8632999999999999e-05 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.103968+0000", 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030692300000000002 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.104214+0000", 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024635400000000001 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.106612+0000", 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00239756 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.109694+0000", 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030822839999999998 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:749 3.7s0 3:e89d09e8:::smithi17529310-19:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e76)", 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:18.078047+0000", 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.016525648, 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026163064, 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 749 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078047+0000", 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078047+0000", 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078048+0000", 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2699999999999999e-07 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078051+0000", 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3289999999999998e-06 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078060+0000", 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3589999999999997e-06 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078066+0000", 2024-09-15T01:33:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078241+0000", 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017526299999999999 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078248+0000", 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9489999999999999e-06 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.080168+0000", 2024-09-15T01:33:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001919825 2024-09-15T01:33:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.080226+0000", 2024-09-15T01:33:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8428999999999999e-05 2024-09-15T01:33:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.103827+0000", 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023600652 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.104087+0000", 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026007 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.104210+0000", 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012296400000000001 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.207 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:23.208 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:33:23.209 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:23.205+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:23.309 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:23.308+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:23.350 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:23.350 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:23.351 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:33:23.351 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:23.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:33:23.410 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:23.409+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:23.459 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:23.460 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:23.460 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:33:23.460 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:33:23.460 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:23.468 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:33:23.511 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:23.510+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:23.570 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:33:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:33:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.5s1 33/31 ECSubWrite(tid=1, reqid=client.4477.0:11, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.389214889000002, 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030602694, 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178301+0000", 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8900000000000001e-07 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178311+0000", 2024-09-15T01:33:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5959999999999993e-06 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208281+0000", 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029970173999999999 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208291+0000", 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3850000000000008e-06 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208407+0000", 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116168 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208422+0000", 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4931e-05 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208519+0000", 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7213999999999995e-05 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208541+0000", 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2249999999999999e-05 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208644+0000", 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102537 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208664+0000", 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.071e-05 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.208904+0000", 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023944 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.5s1 34/31 ECSubRead(tid=6, to_read={3:af8bec4d:::smithi17529310-24:head=0,4096,0}, subchunks={3:af8bec4d:::smithi17529310-24:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.210749+0000", 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.356767013000002, 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033380170000000001, 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210749+0000", 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210749+0000", 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210750+0000", 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6000000000000004e-07 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.210754+0000", 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7049999999999998e-06 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.243939+0000", 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033185092999999999 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.243951+0000", 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1864e-05 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.244010+0000", 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8536000000000003e-05 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.244129+0000", 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119412 2024-09-15T01:33:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/31 ECSubWrite(tid=22, reqid=client.4477.0:113, at_version=34'10, trim_to=0'0, roll_forward_to=34'7))", 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.312297139999998, 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035418835000000003, 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255219+0000", 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.255233+0000", 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3815000000000001e-05 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.289999+0000", 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.034765375000000001 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290004+0000", 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290062+0000", 2024-09-15T01:33:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7763999999999998e-05 2024-09-15T01:33:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290072+0000", 2024-09-15T01:33:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7829999999999993e-06 2024-09-15T01:33:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290165+0000", 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3039000000000001e-05 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290192+0000", 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7449000000000001e-05 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290303+0000", 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110253 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290375+0000", 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2855999999999995e-05 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.290638+0000", 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026231300000000002 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:128 3.4s0 3:299c4018:::smithi17529310-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.275429+0000", 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.292087037999998, 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.10370515499999999, 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 128 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275429+0000", 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275429+0000", 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275430+0000", 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1099999999999995e-07 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275434+0000", 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.42e-06 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275443+0000", 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1330000000000005e-06 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275448+0000", 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3769999999999998e-06 2024-09-15T01:33:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.276441+0000", 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099363500000000009 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.276580+0000", 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013892099999999999 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277288+0000", 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070808500000000001 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277385+0000", 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6824999999999995e-05 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277394+0000", 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0839999999999998e-06 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277434+0000", 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9465999999999998e-05 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277551+0000", 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117162 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379048+0000", 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101497254 2024-09-15T01:33:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379134+0000", 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6081999999999993e-05 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:129 3.4s0 3:299c4018:::smithi17529310-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.292047197000002, 2024-09-15T01:33:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.105118901, 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 129 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275469+0000", 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8999999999999996e-07 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275472+0000", 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6419999999999999e-06 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275480+0000", 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0790000000000001e-06 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.275485+0000", 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4689999999999999e-06 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277457+0000", 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019728160000000001 2024-09-15T01:33:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.277534+0000", 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6073000000000002e-05 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379464+0000", 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.10193010499999999 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.379556+0000", 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2084999999999997e-05 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.380588+0000", 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010320419999999999 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.cs1 34/31 ECSubRead(tid=37, to_read={3:34a45b5f:::smithi17529310-48:head=0,4096,0}, subchunks={3:34a45b5f:::smithi17529310-48:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:33:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.259681383, 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14311833299999999, 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307835+0000", 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6499999999999999e-07 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307844+0000", 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4810000000000006e-06 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450796+0000", 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14295194899999999 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450806+0000", 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7890000000000006e-06 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450858+0000", 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2577999999999997e-05 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.450953+0000", 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4870999999999993e-05 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:217 3.4s0 3:299c4018:::smithi17529310-43:head [copy-from ver 4] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.208686+0000", 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.358829964000002, 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013624864, 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 217 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208686+0000", 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208686+0000", 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208688+0000", 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3989999999999998e-06 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208709+0000", 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0849999999999999e-05 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208729+0000", 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9681e-05 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208738+0000", 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5350000000000004e-06 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208841+0000", 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103353 2024-09-15T01:33:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208941+0000", 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9986000000000001e-05 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.219554+0000", 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01061337 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.219922+0000", 2024-09-15T01:33:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036728800000000001 2024-09-15T01:33:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.219935+0000", 2024-09-15T01:33:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3689000000000001e-05 2024-09-15T01:33:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220006+0000", 2024-09-15T01:33:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1073000000000002e-05 2024-09-15T01:33:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220034+0000", 2024-09-15T01:33:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7736e-05 2024-09-15T01:33:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.221399+0000", 2024-09-15T01:33:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013649000000000001 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.222311+0000", 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000912004 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:218 3.4s0 3:299c4018:::smithi17529310-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:33:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.358763428000003, 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011546681, 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 218 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208753+0000", 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6799999999999999e-07 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208757+0000", 2024-09-15T01:33:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.941e-06 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208767+0000", 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8509999999999994e-06 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208786+0000", 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8320000000000001e-05 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209130+0000", 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034496100000000002 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209184+0000", 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3703999999999999e-05 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220077+0000", 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010893277999999999 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220195+0000", 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117342 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.220299+0000", 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104516 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs1 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.241172+0000", 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.326343712000003, 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018176115, 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241172+0000", 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241172+0000", 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241173+0000", 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.05e-07 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241195+0000", 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2033000000000001e-05 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241238+0000", 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3507999999999997e-05 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241245+0000", 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2509999999999997e-06 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245590+0000", 2024-09-15T01:33:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043445309999999996 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245599+0000", 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8519999999999993e-06 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246349+0000", 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00075022999999999995 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246362+0000", 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.273e-05 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246399+0000", 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7299000000000003e-05 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246806+0000", 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040714800000000003 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.259348+0000", 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012542328 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs1 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.324250470000003, 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017149162999999999, 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243266+0000", 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243294+0000", 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7868000000000002e-05 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243334+0000", 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9369000000000002e-05 2024-09-15T01:33:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243341+0000", 2024-09-15T01:33:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7009999999999996e-06 2024-09-15T01:33:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246522+0000", 2024-09-15T01:33:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031802359999999999 2024-09-15T01:33:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246530+0000", 2024-09-15T01:33:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5019999999999992e-06 2024-09-15T01:33:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246730+0000", 2024-09-15T01:33:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019988499999999999 2024-09-15T01:33:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246745+0000", 2024-09-15T01:33:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4632e-05 2024-09-15T01:33:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246784+0000", 2024-09-15T01:33:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9072000000000001e-05 2024-09-15T01:33:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246850+0000", 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699000000000006e-05 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.260415+0000", 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013564296 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds5 44/31 ECSubWrite(tid=36, reqid=osd.7.0:36, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:33:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.321525469000001, 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013253133, 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245991+0000", 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6199999999999999e-07 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246013+0000", 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1801e-05 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246123+0000", 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109879 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246132+0000", 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8670000000000006e-06 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246240+0000", 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107211 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246248+0000", 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8480000000000007e-06 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246452+0000", 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020319399999999999 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246461+0000", 2024-09-15T01:33:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246492+0000", 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1124999999999998e-05 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246831+0000", 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033878400000000001 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.259244+0000", 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012412444 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.246909+0000", 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.320607441, 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013946906, 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246909+0000", 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246909+0000", 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246910+0000", 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.16e-07 2024-09-15T01:33:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246933+0000", 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3884e-05 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246981+0000", 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7194000000000002e-05 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.246988+0000", 2024-09-15T01:33:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0260000000000002e-06 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247055+0000", 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7216000000000002e-05 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247066+0000", 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1562999999999999e-05 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248672+0000", 2024-09-15T01:33:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001605455 2024-09-15T01:33:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248697+0000", 2024-09-15T01:33:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5588e-05 2024-09-15T01:33:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248730+0000", 2024-09-15T01:33:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2351000000000002e-05 2024-09-15T01:33:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248812+0000", 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2254000000000002e-05 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.260856+0000", 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012043458999999999 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds5 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.320389554000002, 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014066664, 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247127+0000", 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4000000000000001e-07 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247148+0000", 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1339e-05 2024-09-15T01:33:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247188+0000", 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9301000000000002e-05 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247196+0000", 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2789999999999998e-06 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247453+0000", 2024-09-15T01:33:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025692899999999998 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247463+0000", 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0317e-05 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248725+0000", 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012613730000000001 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248732+0000", 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9129999999999997e-06 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248776+0000", 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4280000000000003e-05 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248811+0000", 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5553000000000003e-05 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.261193+0000", 2024-09-15T01:33:23.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01238174 2024-09-15T01:33:23.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.642 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.642 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as3 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:23.642 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:33:23.642 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.319656303000002, 2024-09-15T01:33:23.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012019658000000001, 2024-09-15T01:33:23.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247860+0000", 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247861+0000", 2024-09-15T01:33:23.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6700000000000002e-07 2024-09-15T01:33:23.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247872+0000", 2024-09-15T01:33:23.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1307e-05 2024-09-15T01:33:23.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247909+0000", 2024-09-15T01:33:23.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7290999999999999e-05 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247917+0000", 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3590000000000002e-06 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248032+0000", 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115566 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248048+0000", 2024-09-15T01:33:23.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6069000000000001e-05 2024-09-15T01:33:23.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248184+0000", 2024-09-15T01:33:23.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013525399999999999 2024-09-15T01:33:23.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248205+0000", 2024-09-15T01:33:23.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1021000000000001e-05 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248244+0000", 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9847e-05 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248260+0000", 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5639000000000001e-05 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.259879+0000", 2024-09-15T01:33:23.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011619338 2024-09-15T01:33:23.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.648 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.648 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:292 3.4s0 3:299c4018:::smithi17529310-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:23.648 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.267708+0000", 2024-09-15T01:33:23.648 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.299808163000002, 2024-09-15T01:33:23.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020077541000000001, 2024-09-15T01:33:23.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 292 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267708+0000", 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267708+0000", 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267709+0000", 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7800000000000001e-07 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267712+0000", 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-15T01:33:23.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267723+0000", 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1388e-05 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.267729+0000", 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5890000000000002e-06 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269684+0000", 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019553389999999999 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.269888+0000", 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020362599999999999 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.273250+0000", 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033616280000000002 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.273338+0000", 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7990000000000003e-05 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.287785+0000", 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01444779 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:598 3.4s0 3:269bd1ac:::smithi17529310-23:head [append 0~753664] snapc 1e=[1e,1d,1a,19,15] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:06.076859+0000", 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.490656976, 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012447475, 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 598 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.076859+0000", 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.076859+0000", 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.076861+0000", 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-15T01:33:23.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.077713+0000", 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085228799999999998 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.077727+0000", 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3472000000000001e-05 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.077732+0000", 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4299999999999997e-06 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.077778+0000", 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6081000000000001e-05 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.077783+0000", 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9949999999999996e-06 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.078931+0000", 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011474409999999999 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.079059+0000", 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012866 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.080253+0000", 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011939310000000001 2024-09-15T01:33:23.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.080539+0000", 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028544900000000001 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.080551+0000", 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2418e-05 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.080706+0000", 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015443999999999999 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.080727+0000", 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1821e-05 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.089262+0000", 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085342019999999994 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:06.089307+0000", 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5006999999999998e-05 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs1 66/31 ECSubWrite(tid=198, reqid=osd.11.0:198, at_version=66'62, trim_to=0'0, roll_forward_to=59'58))", 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:08.082679+0000", 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.484837002000001, 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010665167, 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:23.662 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082679+0000", 2024-09-15T01:33:23.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082679+0000", 2024-09-15T01:33:23.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082679+0000", 2024-09-15T01:33:23.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4499999999999998e-07 2024-09-15T01:33:23.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082694+0000", 2024-09-15T01:33:23.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4242e-05 2024-09-15T01:33:23.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082914+0000", 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022094500000000001 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082920+0000", 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084323+0000", 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001403191 2024-09-15T01:33:23.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084331+0000", 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3450000000000006e-06 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084767+0000", 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043536399999999999 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084784+0000", 2024-09-15T01:33:23.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6939000000000002e-05 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084820+0000", 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6025000000000002e-05 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084884+0000", 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3848000000000003e-05 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.093344+0000", 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084606630000000002 2024-09-15T01:33:23.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:642 3.4s0 3:269bd1ac:::smithi17529310-23:head [read 0~2293760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:11.515875+0000", 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.051641097999999, 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.97750444800000003, 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 642 2024-09-15T01:33:23.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.515875+0000", 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.515875+0000", 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.515875+0000", 2024-09-15T01:33:23.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7599999999999997e-07 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.515878+0000", 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0149999999999999e-06 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.515885+0000", 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7660000000000001e-06 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.515889+0000", 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8330000000000004e-06 2024-09-15T01:33:23.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.517051+0000", 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011621019999999999 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.517251+0000", 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000199617 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.493379+0000", 2024-09-15T01:33:23.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.97612863900000002 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:666 3.4s0 3:299c4018:::smithi17529310-43:head [append 0~524288] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.238057+0000", 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.329459367, 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010740744, 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 666 2024-09-15T01:33:23.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238057+0000", 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238057+0000", 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238059+0000", 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9460000000000001e-06 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238633+0000", 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00057398900000000001 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238654+0000", 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1778000000000001e-05 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238661+0000", 2024-09-15T01:33:23.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4999999999999996e-06 2024-09-15T01:33:23.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238725+0000", 2024-09-15T01:33:23.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3669000000000003e-05 2024-09-15T01:33:23.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.238866+0000", 2024-09-15T01:33:23.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014093299999999999 2024-09-15T01:33:23.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:23.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.239996+0000", 2024-09-15T01:33:23.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011303800000000001 2024-09-15T01:33:23.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:23.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.240351+0000", 2024-09-15T01:33:23.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035475600000000002 2024-09-15T01:33:23.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:23.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.240365+0000", 2024-09-15T01:33:23.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4279e-05 2024-09-15T01:33:23.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:23.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.240542+0000", 2024-09-15T01:33:23.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017720600000000001 2024-09-15T01:33:23.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:23.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.240564+0000", 2024-09-15T01:33:23.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1753999999999999e-05 2024-09-15T01:33:23.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:23.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.248736+0000", 2024-09-15T01:33:23.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081716800000000006 2024-09-15T01:33:23.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.248797+0000", 2024-09-15T01:33:23.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1873999999999997e-05 2024-09-15T01:33:23.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.678 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.678 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:671 3.4s0 3:299c4018:::smithi17529310-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:23.678 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.249068+0000", 2024-09-15T01:33:23.678 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.318447743, 2024-09-15T01:33:23.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012181770999999999, 2024-09-15T01:33:23.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:23.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:23.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:23.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:23.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:23.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 671 2024-09-15T01:33:23.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.679 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:23.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:23.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.249068+0000", 2024-09-15T01:33:23.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:23.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.249068+0000", 2024-09-15T01:33:23.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:23.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:23.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.249069+0000", 2024-09-15T01:33:23.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.72e-07 2024-09-15T01:33:23.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:23.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.249070+0000", 2024-09-15T01:33:23.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1990000000000001e-06 2024-09-15T01:33:23.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:23.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.249073+0000", 2024-09-15T01:33:23.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4690000000000002e-06 2024-09-15T01:33:23.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:23.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.249078+0000", 2024-09-15T01:33:23.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6779999999999999e-06 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.253529+0000", 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044515609999999997 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.253631+0000", 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101222 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:23.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:23.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.261250+0000", 2024-09-15T01:33:23.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0076193700000000003 2024-09-15T01:33:23.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.683 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:23.683 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:23.683 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:23.685 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-15T01:33:23.685 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:23.682+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:23.785 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:23.784+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:23.826 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:23.826 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:23.826 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:33:23.826 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:23.835 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-15T01:33:23.886 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:23.885+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:23.936 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:23.936 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:23.936 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:33:23.936 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:33:23.936 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:23.944 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-15T01:33:23.987 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:23.985+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:24.035 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:33:24.084 INFO:tasks.rados.rados.0.smithi175.stdout:308: done (1 left) 2024-09-15T01:33:24.084 INFO:tasks.rados.rados.0.smithi175.stdout:310: done (0 left) 2024-09-15T01:33:24.084 INFO:tasks.rados.rados.0.smithi175.stdout:311: snap_remove snap 19 2024-09-15T01:33:24.088 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:33:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:33:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:33:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 34/31 ECSubWrite(tid=7, reqid=client.4477.0:44, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.897288940000003, 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.119995086, 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188423+0000", 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188439+0000", 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5613000000000001e-05 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307929+0000", 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.119490443 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.307936+0000", 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1740000000000003e-06 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308009+0000", 2024-09-15T01:33:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2714999999999998e-05 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308018+0000", 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8019999999999998e-06 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308149+0000", 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130739 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308161+0000", 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2481e-05 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308199+0000", 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7492e-05 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308216+0000", 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7076999999999999e-05 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308418+0000", 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020204899999999999 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 39/38 ECSubWrite(tid=11, reqid=client.4477.0:196, at_version=39'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.879403725000003, 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042772959999999999, 2024-09-15T01:33:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206308+0000", 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8800000000000002e-07 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206341+0000", 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2418e-05 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208467+0000", 2024-09-15T01:33:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021264159999999999 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208477+0000", 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6090000000000007e-06 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208544+0000", 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6804000000000006e-05 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208555+0000", 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0869e-05 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208869+0000", 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031429800000000002 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208897+0000", 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8416999999999999e-05 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208959+0000", 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2201999999999998e-05 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208984+0000", 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4618999999999999e-05 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210585+0000", 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001601056 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 39/38 ECSubWrite(tid=12, reqid=client.4477.0:197, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.208410+0000", 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.877301281000001, 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042832929999999996, 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208410+0000", 2024-09-15T01:33:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208410+0000", 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208411+0000", 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7100000000000011e-07 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208438+0000", 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6424e-05 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209653+0000", 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001215617 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209664+0000", 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0244999999999999e-05 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209722+0000", 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8675000000000003e-05 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209732+0000", 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3470000000000005e-06 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209848+0000", 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011628099999999999 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209861+0000", 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.359e-05 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209921+0000", 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9296000000000003e-05 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210211+0000", 2024-09-15T01:33:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029074999999999999 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212694+0000", 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024820969999999999 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 39/38 ECSubWrite(tid=13, reqid=client.4477.0:198, at_version=39'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:33:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.876137049999997, 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032744699999999998, 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209575+0000", 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2999999999999999e-07 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209623+0000", 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7089999999999998e-05 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211317+0000", 2024-09-15T01:33:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016948569999999999 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211325+0000", 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9699999999999999e-06 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211380+0000", 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4938000000000002e-05 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211389+0000", 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7439999999999996e-06 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211510+0000", 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012121 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211545+0000", 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5209999999999997e-05 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211601+0000", 2024-09-15T01:33:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5541000000000002e-05 2024-09-15T01:33:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211625+0000", 2024-09-15T01:33:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4334000000000001e-05 2024-09-15T01:33:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212849+0000", 2024-09-15T01:33:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012236459999999999 2024-09-15T01:33:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.241466+0000", 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.844245143999999, 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080416059999999998, 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241466+0000", 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241466+0000", 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241467+0000", 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3999999999999998e-07 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.241497+0000", 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0596999999999999e-05 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243396+0000", 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018991819999999999 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243404+0000", 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2119999999999997e-06 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243529+0000", 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012554800000000001 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243541+0000", 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1834000000000001e-05 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243948+0000", 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040691300000000001 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244199+0000", 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025096899999999998 2024-09-15T01:33:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244271+0000", 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1666000000000001e-05 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245015+0000", 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00074422900000000001 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.249508+0000", 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044932160000000004 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.843223586999997, 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056362690000000002, 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242488+0000", 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0699999999999998e-07 2024-09-15T01:33:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242511+0000", 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2926999999999999e-05 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242662+0000", 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015114499999999999 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242671+0000", 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7730000000000005e-06 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242736+0000", 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4651000000000002e-05 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242746+0000", 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0203e-05 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242974+0000", 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022804800000000001 2024-09-15T01:33:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243002+0000", 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7792000000000001e-05 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243044+0000", 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2657999999999997e-05 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243365+0000", 2024-09-15T01:33:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032055899999999999 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248124+0000", 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047592060000000002 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242514+0000", 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.843197322999998, 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090428599999999998, 2024-09-15T01:33:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242514+0000", 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242514+0000", 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242515+0000", 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242548+0000", 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3615000000000003e-05 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242692+0000", 2024-09-15T01:33:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014390900000000001 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242698+0000", 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4160000000000003e-06 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243566+0000", 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086795700000000004 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243574+0000", 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6889999999999993e-06 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243798+0000", 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000223611 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244198+0000", 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040037200000000002 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244270+0000", 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1956e-05 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244333+0000", 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2889e-05 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251557+0000", 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072240569999999999 2024-09-15T01:33:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 44/38 ECSubWrite(tid=20, reqid=osd.0.0:20, at_version=44'12, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.843145464000003, 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052822889999999999, 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242566+0000", 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4600000000000001e-07 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242585+0000", 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9040000000000001e-05 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243433+0000", 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084765999999999997 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243441+0000", 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4179999999999997e-06 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243507+0000", 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5222999999999995e-05 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243517+0000", 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0567000000000001e-05 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243744+0000", 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022733200000000001 2024-09-15T01:33:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244198+0000", 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045302899999999998 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244269+0000", 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1445999999999993e-05 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244295+0000", 2024-09-15T01:33:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5799000000000002e-05 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247848+0000", 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003553629 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243616+0000", 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.842095155999999, 2024-09-15T01:33:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081799560000000004, 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243616+0000", 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243616+0000", 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243617+0000", 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243646+0000", 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.879e-05 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243680+0000", 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4082000000000003e-05 2024-09-15T01:33:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243686+0000", 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1589999999999998e-06 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243983+0000", 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029721700000000001 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243991+0000", 2024-09-15T01:33:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6240000000000002e-06 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244172+0000", 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018103300000000001 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244199+0000", 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7254999999999999e-05 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244271+0000", 2024-09-15T01:33:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.203e-05 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.245044+0000", 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077298299999999998 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.251796+0000", 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006752096 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.247492+0000", 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.838219746999997, 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045361799999999999, 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247492+0000", 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247492+0000", 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247494+0000", 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7230000000000001e-06 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247507+0000", 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3421000000000001e-05 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247558+0000", 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0801999999999999e-05 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247567+0000", 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9889999999999998e-06 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247635+0000", 2024-09-15T01:33:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8623999999999999e-05 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247647+0000", 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1844999999999999e-05 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247812+0000", 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016474600000000001 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247865+0000", 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3183000000000002e-05 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247909+0000", 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3645e-05 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.247928+0000", 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9049e-05 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252028+0000", 2024-09-15T01:33:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041001529999999996 2024-09-15T01:33:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 44/38 ECSubWrite(tid=21, reqid=osd.0.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:33:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.837485790999999, 2024-09-15T01:33:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040969320000000002, 2024-09-15T01:33:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248226+0000", 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248235+0000", 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7180000000000002e-06 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248464+0000", 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022932900000000001 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248473+0000", 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.789e-06 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248516+0000", 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3380999999999997e-05 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248526+0000", 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2969999999999992e-06 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248624+0000", 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8672000000000007e-05 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248645+0000", 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0327e-05 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248697+0000", 2024-09-15T01:33:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2602000000000002e-05 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.248930+0000", 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023236900000000001 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.252323+0000", 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003393063 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 55/31 ECSubWrite(tid=159, reqid=client.4477.0:511, at_version=55'52, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.075075+0000", 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.010636235, 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080575170000000002, 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075075+0000", 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075075+0000", 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075076+0000", 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9999999999999999e-07 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075102+0000", 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6071000000000002e-05 2024-09-15T01:33:24.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075431+0000", 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032964000000000002 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075438+0000", 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8349999999999998e-06 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075493+0000", 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5220999999999999e-05 2024-09-15T01:33:24.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075504+0000", 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0434999999999999e-05 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075781+0000", 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027726299999999998 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075914+0000", 2024-09-15T01:33:24.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132733 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075957+0000", 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3195999999999998e-05 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076225+0000", 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026822800000000002 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.083133+0000", 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069075949999999999 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 55/31 ECSubWrite(tid=160, reqid=client.4477.0:512, at_version=55'53, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.075980+0000", 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.009731605999999, 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011476257, 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075980+0000", 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075980+0000", 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075981+0000", 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.54e-07 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.075999+0000", 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7974000000000001e-05 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076029+0000", 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0642000000000002e-05 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076035+0000", 2024-09-15T01:33:24.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9750000000000004e-06 2024-09-15T01:33:24.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076082+0000", 2024-09-15T01:33:24.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6227999999999999e-05 2024-09-15T01:33:24.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076091+0000", 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2399999999999996e-06 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076185+0000", 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4315999999999996e-05 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076211+0000", 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5728000000000002e-05 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076263+0000", 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1900000000000001e-05 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076277+0000", 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4239999999999999e-05 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.087456+0000", 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01117916 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 55/31 ECSubWrite(tid=161, reqid=client.4477.0:513, at_version=55'54, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.076704+0000", 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.009007983, 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010911452, 2024-09-15T01:33:24.162 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076704+0000", 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076704+0000", 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076704+0000", 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076723+0000", 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8493000000000001e-05 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076754+0000", 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1479999999999997e-05 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076760+0000", 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.105e-06 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076815+0000", 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5061e-05 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076824+0000", 2024-09-15T01:33:24.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7110000000000001e-06 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076921+0000", 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7176999999999995e-05 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.076950+0000", 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8606999999999999e-05 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.077617+0000", 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00066715999999999998 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.077636+0000", 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8779000000000001e-05 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.087615+0000", 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0099791570000000007 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 55/31 ECSubWrite(tid=162, reqid=client.4477.0:513, at_version=0'0, trim_to=0'0, roll_forward_to=55'54))", 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.078911+0000", 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.006800806000001, 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089080270000000007, 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078911+0000", 2024-09-15T01:33:24.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078911+0000", 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078912+0000", 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.088e-06 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078924+0000", 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2592000000000001e-05 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078964+0000", 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9545999999999997e-05 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.078973+0000", 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7490000000000004e-06 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079047+0000", 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3755000000000001e-05 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079057+0000", 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0383e-05 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079195+0000", 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013842999999999999 2024-09-15T01:33:24.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079269+0000", 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3774000000000002e-05 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079488+0000", 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021939400000000001 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.079506+0000", 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7153999999999999e-05 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.087819+0000", 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083131620000000007 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 66/31 ECSubWrite(tid=198, reqid=osd.11.0:198, at_version=66'62, trim_to=0'0, roll_forward_to=59'58))", 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:08.080233+0000", 2024-09-15T01:33:24.173 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.005478828000001, 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040668559999999998, 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.080233+0000", 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.080233+0000", 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.080233+0000", 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8599999999999999e-07 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.080254+0000", 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605e-05 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.080362+0000", 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010848 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.080367+0000", 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1540000000000001e-06 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082282+0000", 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001915157 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082294+0000", 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.135e-05 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082550+0000", 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025634600000000002 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.082593+0000", 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2518999999999997e-05 2024-09-15T01:33:24.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.083323+0000", 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073059700000000004 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.083347+0000", 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3561e-05 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:08.084300+0000", 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00095280099999999997 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 71/31 ECSubWrite(tid=252, reqid=client.4477.0:699, at_version=71'73, trim_to=0'0, roll_forward_to=71'71))", 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.312764+0000", 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.772947273, 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0067010990000000003, 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.312764+0000", 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.312764+0000", 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.312766+0000", 2024-09-15T01:33:24.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6649999999999999e-06 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.314419+0000", 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016526449999999999 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.314466+0000", 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6901999999999999e-05 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.314476+0000", 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0811e-05 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.314538+0000", 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1358000000000004e-05 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.314550+0000", 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2405000000000001e-05 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.315054+0000", 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00050381899999999997 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.315091+0000", 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6953999999999999e-05 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.315490+0000", 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039952100000000001 2024-09-15T01:33:24.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.315515+0000", 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4343999999999999e-05 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319465+0000", 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039506749999999998 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs5 71/31 ECSubWrite(tid=253, reqid=client.4477.0:699, at_version=0'0, trim_to=0'0, roll_forward_to=71'73))", 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.316660+0000", 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.769051996, 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003192664, 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316660+0000", 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316660+0000", 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316660+0000", 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5199999999999995e-07 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316687+0000", 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6944000000000002e-05 2024-09-15T01:33:24.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316731+0000", 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3507999999999997e-05 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316739+0000", 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3650000000000004e-06 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316819+0000", 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9528000000000002e-05 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316833+0000", 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4319e-05 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.316982+0000", 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000148567 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.317006+0000", 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4604999999999999e-05 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.317197+0000", 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019090599999999999 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.317215+0000", 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7759000000000001e-05 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319852+0000", 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026373109999999998 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs3 71/69 ECSubWrite(tid=179, reqid=client.4477.0:706, at_version=71'25, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.319734+0000", 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.765977964999999, 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035915090000000001, 2024-09-15T01:33:24.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319734+0000", 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319734+0000", 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319735+0000", 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1939999999999999e-06 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319844+0000", 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109064 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319932+0000", 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8086999999999999e-05 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.319940+0000", 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2330000000000006e-06 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.320028+0000", 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7837000000000006e-05 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.320042+0000", 2024-09-15T01:33:24.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4089e-05 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.320346+0000", 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030355600000000002 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.320376+0000", 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0431999999999999e-05 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.320627+0000", 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025115000000000001 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321467+0000", 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083993499999999999 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.323325+0000", 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018579320000000001 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s4 71/69 ECSubWrite(tid=73, reqid=client.4477.0:701, at_version=71'45, trim_to=0'0, roll_forward_to=70'43))", 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.321927+0000", 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.763784077, 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0074903310000000002, 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321927+0000", 2024-09-15T01:33:24.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321927+0000", 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321930+0000", 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1449999999999998e-06 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:24.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322708+0000", 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077836300000000001 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322764+0000", 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6158e-05 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322784+0000", 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9809e-05 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322869+0000", 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4559000000000001e-05 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322880+0000", 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1922e-05 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.323756+0000", 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087522499999999998 2024-09-15T01:33:24.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.323799+0000", 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3216000000000001e-05 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.324311+0000", 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000512132 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.324366+0000", 2024-09-15T01:33:24.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5204999999999997e-05 2024-09-15T01:33:24.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:24.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:24.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:24.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.329418+0000", 2024-09-15T01:33:24.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050515969999999997 2024-09-15T01:33:24.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.201 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:24.201 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:24.201 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:24.202 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:33:24.203 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:24.085+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:24.204 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:33:24.204 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:24.204 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:24.205 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:24.205 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:24.205 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:24.201+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:24.206 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:24.206 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:24.224 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:24.244 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:24.244 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:24.263 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:24.263 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:24.304 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:24.302+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:24.316 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:24.316 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:24.324 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:33:24.379 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:24.379 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:24.399 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:24.405 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:24.404+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:24.417 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:24.417 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:24.421 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:24.421 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:24.430 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:33:24.436 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:24.436 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:24.455 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:24.455 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:24.475 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:24.475 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:24.493 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:24.506 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:24.505+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:24.512 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:24.513 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:24.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:24.517+0000 7fefb94f8640 0 -- 172.21.15.42:0/760233043 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7fefb41949c0 msgr2=0x7fefb4196da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:24.526 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:24.526 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:24.534 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:33:24.538 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:24.538 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:24.557 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:24.609 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:24.608+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:24.630 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:24.630 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:24.639 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:33:24.707 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:24.706+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:24.735 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:24.735 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:24.743 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:33:24.808 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:24.806+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:24.841 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:24.841 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:24.849 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:33:24.933 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:24.931+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:24.986 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:24.986 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:24.994 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:33:25.035 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:25.034+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:25.086 INFO:tasks.rados.rados.0.smithi175.stdout:311: done (0 left) 2024-09-15T01:33:25.086 INFO:tasks.rados.rados.0.smithi175.stdout:312: read oid 17 snap -1 2024-09-15T01:33:25.086 INFO:tasks.rados.rados.0.smithi175.stdout:312: expect (ObjNum 126 snap 20 seq_num 126) 2024-09-15T01:33:25.087 INFO:tasks.rados.rados.0.smithi175.stdout:313: snap_create 2024-09-15T01:33:25.089 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-15T01:33:25.110 INFO:tasks.rados.rados.0.smithi175.stdout:312: expect (ObjNum 126 snap 20 seq_num 126) 2024-09-15T01:33:25.110 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:25.109+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:25.139 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:25.140 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:25.148 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:33:25.211 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:25.210+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:25.244 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:25.244 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:25.252 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-15T01:33:25.312 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:25.310+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:25.348 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:25.349 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:25.357 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-15T01:33:25.413 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:25.412+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:25.455 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:25.455 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:25.464 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-15T01:33:25.514 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:25.512+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:25.559 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:25.559 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:25.567 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:33:25.614 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:25.613+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:25.677 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:25.677 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:25.686 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:33:25.716 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:25.714+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:25.784 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:25.785 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:25.794 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:33:25.837 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:25.835+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:25.890 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:25.891 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:25.899 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:33:25.917 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:25.915+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:26.019 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:26.017+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:26.036 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:26.036 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:26.044 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:33:26.089 INFO:tasks.rados.rados.0.smithi175.stdout:312: done (1 left) 2024-09-15T01:33:26.089 INFO:tasks.rados.rados.0.smithi175.stdout:313: done (0 left) 2024-09-15T01:33:26.089 INFO:tasks.rados.rados.0.smithi175.stdout:314: snap_remove snap 21 2024-09-15T01:33:26.120 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:26.117+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:26.143 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:26.143 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:26.153 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:33:26.237 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:26.234+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:26.289 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:26.289 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:26.298 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:33:26.321 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:26.320+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:26.422 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:26.420+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:26.435 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:26.436 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:26.444 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:33:26.523 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:26.521+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:26.541 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:26.541 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:26.549 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:33:26.569 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:33:26.624 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:26.622+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:26.646 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:26.646 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:26.654 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-15T01:33:26.707 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:26.707 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:26.724 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:26.723+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:26.727 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:26.727 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:26.747 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:26.750 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:26.750 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:26.759 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-15T01:33:26.767 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:26.767 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:26.785 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:26.786 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:26.805 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:26.805 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:26.824 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:26.825 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:26.823+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:26.844 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:26.845 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:26.856 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:26.856 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:26.863 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:26.863 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:26.865 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-15T01:33:26.883 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:26.883 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:26.902 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:26.902 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:26.921 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:26.922 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:26.926 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:26.924+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:26.940 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:26.940 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:26.959 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:26.960 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:26.962 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:26.962 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:26.971 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:33:26.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:26.978+0000 7f5a0580a640 0 -- 172.21.15.42:0/3138212208 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7f5a00194a00 msgr2=0x7f5a00196e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:27.027 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:27.025+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:27.080 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:27.081 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:27.081 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:27.081 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:27.090 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:33:27.099 INFO:tasks.rados.rados.0.smithi175.stdout:314: done (0 left) 2024-09-15T01:33:27.099 INFO:tasks.rados.rados.0.smithi175.stdout:315: rollback oid 12 current snap is 28 2024-09-15T01:33:27.099 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 12 to 25 2024-09-15T01:33:27.099 INFO:tasks.rados.rados.0.smithi175.stdout:316: rollback oid 14 current snap is 28 2024-09-15T01:33:27.099 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 14 to 17 2024-09-15T01:33:27.099 INFO:tasks.rados.rados.0.smithi175.stdout:317: setattr oid 41 current snap is 28 2024-09-15T01:33:27.100 INFO:tasks.rados.rados.0.smithi175.stdout:318: read oid 46 snap 24 2024-09-15T01:33:27.101 INFO:tasks.rados.rados.0.smithi175.stdout:318: expect (ObjNum 135 snap 24 seq_num 135) 2024-09-15T01:33:27.101 INFO:tasks.rados.rados.0.smithi175.stdout:319: read oid 3 snap 24 2024-09-15T01:33:27.101 INFO:tasks.rados.rados.0.smithi175.stdout:319: expect (ObjNum 118 snap 19 seq_num 118) 2024-09-15T01:33:27.101 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 45 current snap is 28 2024-09-15T01:33:27.101 INFO:tasks.rados.rados.0.smithi175.stdout:320: seq_num 138 ranges {1622016=770048,2392064=540672,2932736=753664,3686400=262144} 2024-09-15T01:33:27.114 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:33:27.116 INFO:tasks.rados.rados.0.smithi175.stdout:320: writing smithi17529310-45 from 1622016 to 2392064 tid 1 2024-09-15T01:33:27.120 INFO:tasks.rados.rados.0.smithi175.stdout:320: writing smithi17529310-45 from 2392064 to 2932736 tid 2 2024-09-15T01:33:27.125 INFO:tasks.rados.rados.0.smithi175.stdout:320: writing smithi17529310-45 from 2932736 to 3686400 tid 3 2024-09-15T01:33:27.127 INFO:tasks.rados.rados.0.smithi175.stdout:320: writing smithi17529310-45 from 3686400 to 3948544 tid 4 2024-09-15T01:33:27.127 INFO:tasks.rados.rados.0.smithi175.stdout:316: finishing rollback tid 1 to smithi17529310-14 2024-09-15T01:33:27.127 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 41 v 68 (ObjNum 177 snap 0 seq_num 0) dirty exists 2024-09-15T01:33:27.127 INFO:tasks.rados.rados.0.smithi175.stdout:316: finishing rollback tid 2 to smithi17529310-14 2024-09-15T01:33:27.127 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 14 v 67 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-15T01:33:27.127 INFO:tasks.rados.rados.0.smithi175.stdout:315: finishing rollback tid 0 to smithi17529310-12 2024-09-15T01:33:27.127 INFO:tasks.rados.rados.0.smithi175.stdout:315: finishing rollback tid 1 to smithi17529310-12 2024-09-15T01:33:27.127 INFO:tasks.rados.rados.0.smithi175.stdout:315: finishing rollback tid 2 to smithi17529310-12 2024-09-15T01:33:27.127 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 12 v 103 (ObjNum 136 snap 24 seq_num 136) dirty exists 2024-09-15T01:33:27.128 INFO:tasks.rados.rados.0.smithi175.stdout:315: done (5 left) 2024-09-15T01:33:27.128 INFO:tasks.rados.rados.0.smithi175.stdout:316: done (4 left) 2024-09-15T01:33:27.128 INFO:tasks.rados.rados.0.smithi175.stdout:317: done (3 left) 2024-09-15T01:33:27.128 INFO:tasks.rados.rados.0.smithi175.stdout:321: copy_from oid 21 from oid 7 current snap is 28 2024-09-15T01:33:27.128 INFO:tasks.rados.rados.0.smithi175.stdout:322: rollback oid 13 current snap is 28 2024-09-15T01:33:27.128 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 13 to 25 2024-09-15T01:33:27.128 INFO:tasks.rados.rados.0.smithi175.stdout:323: snap_create 2024-09-15T01:33:27.130 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:27.127+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:27.133 INFO:tasks.rados.rados.0.smithi175.stdout:320: finishing write tid 1 to smithi17529310-45 2024-09-15T01:33:27.139 INFO:tasks.rados.rados.0.smithi175.stdout:319: expect (ObjNum 118 snap 19 seq_num 118) 2024-09-15T01:33:27.166 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 9] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-15T01:33:27.166 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:33:27.166 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-15T01:33:27.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-15T01:33:27.215 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:27.215 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:27.215 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:27.216 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:27.216 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:27.224 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:33:27.266 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:27.264+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:27.277 INFO:tasks.rados.rados.0.smithi175.stdout:320: finishing write tid 2 to smithi17529310-45 2024-09-15T01:33:27.277 INFO:tasks.rados.rados.0.smithi175.stdout:320: finishing write tid 3 to smithi17529310-45 2024-09-15T01:33:27.277 INFO:tasks.rados.rados.0.smithi175.stdout:320: finishing write tid 4 to smithi17529310-45 2024-09-15T01:33:27.277 INFO:tasks.rados.rados.0.smithi175.stdout:320: finishing write tid 5 to smithi17529310-45 2024-09-15T01:33:27.277 INFO:tasks.rados.rados.0.smithi175.stdout:320: finishing write tid 6 to smithi17529310-45 2024-09-15T01:33:27.277 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 45 v 83 (ObjNum 138 snap 28 seq_num 138) dirty exists 2024-09-15T01:33:27.277 INFO:tasks.rados.rados.0.smithi175.stdout:320: left oid 45 (ObjNum 138 snap 28 seq_num 138) 2024-09-15T01:33:27.277 INFO:tasks.rados.rados.0.smithi175.stdout:318: expect (ObjNum 135 snap 24 seq_num 135) 2024-09-15T01:33:27.312 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-15T01:33:27.330 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:27.329+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:27.364 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:192 3.as0 3:5e9da7d3:::smithi17529310-1:head [copy-from ver 6] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 48.169202464000001, 2024-09-15T01:33:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01403071, 2024-09-15T01:33:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 192 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192423+0000", 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3140000000000002e-06 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192431+0000", 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2320000000000001e-06 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192444+0000", 2024-09-15T01:33:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3239e-05 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192454+0000", 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2439999999999999e-06 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192781+0000", 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032732699999999998 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192904+0000", 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012317900000000001 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.201873+0000", 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089691839999999998 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202377+0000", 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00050339000000000004 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202392+0000", 2024-09-15T01:33:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5201999999999999e-05 2024-09-15T01:33:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202869+0000", 2024-09-15T01:33:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047669800000000001 2024-09-15T01:33:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202888+0000", 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9202e-05 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203431+0000", 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00054287499999999995 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.206451+0000", 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003020624 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:193 3.as0 3:5e9da7d3:::smithi17529310-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192463+0000", 2024-09-15T01:33:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 48.16916003, 2024-09-15T01:33:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010399754000000001, 2024-09-15T01:33:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 193 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192463+0000", 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192463+0000", 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192464+0000", 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192466+0000", 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5390000000000002e-06 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192476+0000", 2024-09-15T01:33:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6549999999999999e-06 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192483+0000", 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9550000000000003e-06 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.193098+0000", 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00061465200000000004 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.193155+0000", 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7608999999999997e-05 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202537+0000", 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093813579999999994 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202652+0000", 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115473 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202863+0000", 2024-09-15T01:33:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000210826 2024-09-15T01:33:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:216 3.as0 3:5d5028c6:::smithi17529310-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:41.208948+0000", 2024-09-15T01:33:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.152675420000001, 2024-09-15T01:33:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0098072100000000002, 2024-09-15T01:33:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 216 2024-09-15T01:33:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208948+0000", 2024-09-15T01:33:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208948+0000", 2024-09-15T01:33:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208948+0000", 2024-09-15T01:33:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999997e-07 2024-09-15T01:33:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208950+0000", 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1710000000000001e-06 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208958+0000", 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6599999999999995e-06 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.208963+0000", 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4449999999999998e-06 2024-09-15T01:33:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210679+0000", 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017165189999999999 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.210762+0000", 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2719000000000005e-05 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.217447+0000", 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0066855090000000001 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.217557+0000", 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011001 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.218755+0000", 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011976630000000001 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.114648252999999, 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013669871, 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246976+0000", 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9800000000000002e-07 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247000+0000", 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4314000000000001e-05 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247045+0000", 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4938000000000002e-05 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247053+0000", 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:33:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247132+0000", 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8799000000000002e-05 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247142+0000", 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0511e-05 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247515+0000", 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037233299999999997 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247546+0000", 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1102000000000001e-05 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247585+0000", 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9152000000000001e-05 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247603+0000", 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.260645+0000", 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013041662000000001 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 44/31 ECSubWrite(tid=38, reqid=osd.7.0:37, at_version=0'0, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.251749+0000", 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.109873993000001, 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0092966969999999996, 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251749+0000", 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251749+0000", 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251750+0000", 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6899999999999996e-07 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251765+0000", 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.489e-05 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251796+0000", 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0972000000000001e-05 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251801+0000", 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3000000000000001e-06 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251845+0000", 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3900999999999999e-05 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251851+0000", 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3380000000000001e-06 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.251966+0000", 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011479700000000001 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252286+0000", 2024-09-15T01:33:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031985800000000001 2024-09-15T01:33:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252343+0000", 2024-09-15T01:33:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6909e-05 2024-09-15T01:33:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.252374+0000", 2024-09-15T01:33:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0596000000000003e-05 2024-09-15T01:33:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.261046+0000", 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0086723670000000003 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=54, reqid=client.4477.0:516, at_version=55'44, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.074223+0000", 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 30.287400617999999, 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089159619999999995, 2024-09-15T01:33:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074223+0000", 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074223+0000", 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074224+0000", 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0860000000000001e-06 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.074256+0000", 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2258000000000002e-05 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075603+0000", 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001347139 2024-09-15T01:33:27.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075620+0000", 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7198999999999999e-05 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075708+0000", 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8177999999999995e-05 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075730+0000", 2024-09-15T01:33:27.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1469000000000001e-05 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076062+0000", 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033251199999999998 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076077+0000", 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4523e-05 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076128+0000", 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1371e-05 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076153+0000", 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4349e-05 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083139+0000", 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0069858780000000001 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.399 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=55, reqid=client.4477.0:517, at_version=55'45, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 30.286090745999999, 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016315308000000001, 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075533+0000", 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9199999999999999e-07 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075557+0000", 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3264e-05 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076900+0000", 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013436749999999999 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076916+0000", 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5302999999999999e-05 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076970+0000", 2024-09-15T01:33:27.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4265999999999998e-05 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076980+0000", 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0521e-05 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077104+0000", 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012322599999999999 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077118+0000", 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4742e-05 2024-09-15T01:33:27.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077164+0000", 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5972e-05 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077228+0000", 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3291000000000001e-05 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.091848+0000", 2024-09-15T01:33:27.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014620055999999999 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=56, reqid=client.4477.0:518, at_version=55'46, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 30.284769110999999, 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015201957, 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076855+0000", 2024-09-15T01:33:27.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.057e-06 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076876+0000", 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0874e-05 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079870+0000", 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0029943259999999998 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079882+0000", 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2026e-05 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079958+0000", 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5263000000000001e-05 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079969+0000", 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.145e-05 2024-09-15T01:33:27.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.080116+0000", 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014736600000000001 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.080133+0000", 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6776e-05 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082691+0000", 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025575229999999999 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082740+0000", 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9373000000000003e-05 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.092056+0000", 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093159230000000003 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.411 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.411 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=57, reqid=client.4477.0:518, at_version=0'0, trim_to=0'0, roll_forward_to=55'46))", 2024-09-15T01:33:27.411 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:27.411 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 30.277845383999999, 2024-09-15T01:33:27.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.77065714699999999, 2024-09-15T01:33:27.411 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.411 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.411 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083779+0000", 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083792+0000", 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3478e-05 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853926+0000", 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.77013421000000004 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853932+0000", 2024-09-15T01:33:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2820000000000004e-06 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853992+0000", 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9886999999999998e-05 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854001+0000", 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3700000000000001e-06 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854114+0000", 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000112651 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854152+0000", 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8518e-05 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854203+0000", 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1288000000000001e-05 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854223+0000", 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9884000000000001e-05 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854435+0000", 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021157399999999999 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.416 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s3 56/31 ECSubWrite(tid=115, reqid=osd.14.0:115, at_version=56'45, trim_to=0'0, roll_forward_to=54'41))", 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.079178+0000", 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.282445424999999, 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093555189999999996, 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079178+0000", 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079178+0000", 2024-09-15T01:33:27.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079179+0000", 2024-09-15T01:33:27.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1299999999999998e-07 2024-09-15T01:33:27.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079210+0000", 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1012000000000001e-05 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079243+0000", 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3686000000000002e-05 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079250+0000", 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5490000000000003e-06 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079312+0000", 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2397000000000006e-05 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.079324+0000", 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1362999999999999e-05 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.081543+0000", 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022197779999999999 2024-09-15T01:33:27.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082438+0000", 2024-09-15T01:33:27.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00089462899999999998 2024-09-15T01:33:27.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082489+0000", 2024-09-15T01:33:27.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1326999999999998e-05 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.082657+0000", 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016720600000000001 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.088533+0000", 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0058766590000000002 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.422 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s3 66/31 ECSubWrite(tid=130, reqid=osd.14.0:130, at_version=66'56, trim_to=0'0, roll_forward_to=60'52))", 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.280828387, 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016749074999999999, 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8199999999999999e-07 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080811+0000", 2024-09-15T01:33:27.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6435000000000001e-05 2024-09-15T01:33:27.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082265+0000", 2024-09-15T01:33:27.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001453415 2024-09-15T01:33:27.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082270+0000", 2024-09-15T01:33:27.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9470000000000002e-06 2024-09-15T01:33:27.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082610+0000", 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00034021700000000003 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082618+0000", 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3969999999999994e-06 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084267+0000", 2024-09-15T01:33:27.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001649069 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084292+0000", 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4125999999999999e-05 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084507+0000", 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0002154 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084539+0000", 2024-09-15T01:33:27.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2208000000000001e-05 2024-09-15T01:33:27.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097544+0000", 2024-09-15T01:33:27.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013004679 2024-09-15T01:33:27.428 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.428 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.428 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 66/54 ECSubWrite(tid=240, reqid=osd.13.0:240, at_version=66'36, trim_to=0'0, roll_forward_to=56'33))", 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.280757671, 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020188490999999999, 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:27.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:27.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4600000000000001e-07 2024-09-15T01:33:27.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080879+0000", 2024-09-15T01:33:27.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3557000000000001e-05 2024-09-15T01:33:27.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082859+0000", 2024-09-15T01:33:27.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019801799999999998 2024-09-15T01:33:27.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082865+0000", 2024-09-15T01:33:27.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8309999999999997e-06 2024-09-15T01:33:27.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083375+0000", 2024-09-15T01:33:27.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051010200000000002 2024-09-15T01:33:27.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083383+0000", 2024-09-15T01:33:27.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7740000000000004e-06 2024-09-15T01:33:27.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085762+0000", 2024-09-15T01:33:27.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023785339999999999 2024-09-15T01:33:27.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086258+0000", 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00049678099999999996 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086290+0000", 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1612999999999997e-05 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086368+0000", 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8417000000000005e-05 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101054+0000", 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014685556000000001 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=62, reqid=osd.7.0:62, at_version=66'52, trim_to=0'0, roll_forward_to=58'50))", 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081340+0000", 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.280282852999999, 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016057799000000001, 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.435 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081340+0000", 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081340+0000", 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081341+0000", 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3900000000000004e-07 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081364+0000", 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3078000000000002e-05 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082876+0000", 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001512298 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082880+0000", 2024-09-15T01:33:27.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1150000000000004e-06 2024-09-15T01:33:27.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082998+0000", 2024-09-15T01:33:27.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117386 2024-09-15T01:33:27.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083007+0000", 2024-09-15T01:33:27.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4849999999999998e-06 2024-09-15T01:33:27.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084480+0000", 2024-09-15T01:33:27.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014727799999999999 2024-09-15T01:33:27.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084509+0000", 2024-09-15T01:33:27.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8878e-05 2024-09-15T01:33:27.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084575+0000", 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5848999999999999e-05 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085049+0000", 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047466899999999999 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097398+0000", 2024-09-15T01:33:27.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012348622 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=63, reqid=osd.7.0:63, at_version=66'54, trim_to=0'0, roll_forward_to=58'50))", 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.280255615000002, 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017023607, 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081386+0000", 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952e-05 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083054+0000", 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001668295 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083059+0000", 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9359999999999998e-06 2024-09-15T01:33:27.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084544+0000", 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001484983 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084582+0000", 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7764e-05 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084807+0000", 2024-09-15T01:33:27.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022473099999999999 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084975+0000", 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000167997 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086254+0000", 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001279608 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086318+0000", 2024-09-15T01:33:27.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3590000000000006e-05 2024-09-15T01:33:27.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.098391+0000", 2024-09-15T01:33:27.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012073421000000001 2024-09-15T01:33:27.446 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.446 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.446 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.447 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 66/31 ECSubWrite(tid=241, reqid=osd.13.0:241, at_version=66'51, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:27.447 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081392+0000", 2024-09-15T01:33:27.447 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.280230787000001, 2024-09-15T01:33:27.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017955004, 2024-09-15T01:33:27.447 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.447 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.447 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081392+0000", 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081392+0000", 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081393+0000", 2024-09-15T01:33:27.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7399999999999999e-07 2024-09-15T01:33:27.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081416+0000", 2024-09-15T01:33:27.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3807000000000001e-05 2024-09-15T01:33:27.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083070+0000", 2024-09-15T01:33:27.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001653416 2024-09-15T01:33:27.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083073+0000", 2024-09-15T01:33:27.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.58e-06 2024-09-15T01:33:27.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083228+0000", 2024-09-15T01:33:27.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000154671 2024-09-15T01:33:27.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083236+0000", 2024-09-15T01:33:27.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1960000000000008e-06 2024-09-15T01:33:27.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085387+0000", 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021502230000000002 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086257+0000", 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00087064300000000005 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086289+0000", 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2048000000000002e-05 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086344+0000", 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5031000000000002e-05 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.099347+0000", 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013003115000000001 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.455 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 66/31 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=66'53, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:27.455 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.083317+0000", 2024-09-15T01:33:27.455 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.278306101999998, 2024-09-15T01:33:27.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017650290999999999, 2024-09-15T01:33:27.455 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.455 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.455 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083317+0000", 2024-09-15T01:33:27.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083317+0000", 2024-09-15T01:33:27.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083318+0000", 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9100000000000004e-07 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083337+0000", 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9488e-05 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083371+0000", 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3522000000000002e-05 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083382+0000", 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1715999999999999e-05 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085431+0000", 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020484309999999999 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085439+0000", 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3389999999999993e-06 2024-09-15T01:33:27.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085612+0000", 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017278700000000001 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086258+0000", 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00064593500000000004 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086290+0000", 2024-09-15T01:33:27.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1813999999999997e-05 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086355+0000", 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5405000000000001e-05 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.100967+0000", 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014612263 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.460 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.461 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=64, reqid=client.4477.0:609, at_version=66'56, trim_to=0'0, roll_forward_to=66'52))", 2024-09-15T01:33:27.461 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.087506+0000", 2024-09-15T01:33:27.461 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.274117180000001, 2024-09-15T01:33:27.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014356819999999999, 2024-09-15T01:33:27.461 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.461 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.461 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087506+0000", 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087506+0000", 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087507+0000", 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.144e-06 2024-09-15T01:33:27.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087633+0000", 2024-09-15T01:33:27.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012627100000000001 2024-09-15T01:33:27.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087682+0000", 2024-09-15T01:33:27.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8551000000000003e-05 2024-09-15T01:33:27.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087690+0000", 2024-09-15T01:33:27.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5979999999999999e-06 2024-09-15T01:33:27.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087747+0000", 2024-09-15T01:33:27.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7376000000000001e-05 2024-09-15T01:33:27.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087758+0000", 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1065e-05 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.088106+0000", 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000347766 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.088114+0000", 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3850000000000002e-06 2024-09-15T01:33:27.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091745+0000", 2024-09-15T01:33:27.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003630968 2024-09-15T01:33:27.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091795+0000", 2024-09-15T01:33:27.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9922000000000001e-05 2024-09-15T01:33:27.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101863+0000", 2024-09-15T01:33:27.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010067774 2024-09-15T01:33:27.467 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.467 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.467 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.467 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 66/31 ECSubWrite(tid=244, reqid=osd.13.0:242, at_version=0'0, trim_to=0'0, roll_forward_to=66'53))", 2024-09-15T01:33:27.467 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.089639+0000", 2024-09-15T01:33:27.467 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.271983732999999, 2024-09-15T01:33:27.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012039856, 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089639+0000", 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089639+0000", 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089640+0000", 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089649+0000", 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0459999999999994e-06 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089681+0000", 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1588999999999999e-05 2024-09-15T01:33:27.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089687+0000", 2024-09-15T01:33:27.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9819999999999996e-06 2024-09-15T01:33:27.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089743+0000", 2024-09-15T01:33:27.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5841999999999998e-05 2024-09-15T01:33:27.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089751+0000", 2024-09-15T01:33:27.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3790000000000006e-06 2024-09-15T01:33:27.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089878+0000", 2024-09-15T01:33:27.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012716299999999999 2024-09-15T01:33:27.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091756+0000", 2024-09-15T01:33:27.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018777080000000001 2024-09-15T01:33:27.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091830+0000", 2024-09-15T01:33:27.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4288000000000004e-05 2024-09-15T01:33:27.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091848+0000", 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7448e-05 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101679+0000", 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0098317509999999997 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.473 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 66/54 ECSubWrite(tid=243, reqid=osd.13.0:240, at_version=0'0, trim_to=0'0, roll_forward_to=66'36))", 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.271971304000001, 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01224519, 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:27.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.48e-07 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089655+0000", 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089726+0000", 2024-09-15T01:33:27.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0436000000000001e-05 2024-09-15T01:33:27.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089731+0000", 2024-09-15T01:33:27.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.164e-06 2024-09-15T01:33:27.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089770+0000", 2024-09-15T01:33:27.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9110999999999999e-05 2024-09-15T01:33:27.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089777+0000", 2024-09-15T01:33:27.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2599999999999999e-06 2024-09-15T01:33:27.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.090039+0000", 2024-09-15T01:33:27.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000261506 2024-09-15T01:33:27.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091756+0000", 2024-09-15T01:33:27.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001717704 2024-09-15T01:33:27.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091831+0000", 2024-09-15T01:33:27.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4177000000000005e-05 2024-09-15T01:33:27.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091855+0000", 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4300000000000001e-05 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101897+0000", 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010042163 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.as0 76/69 ECSubWriteReply(tid=60, last_complete=76'39, committed=1, applied=1))", 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:18.085798+0000", 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2758252050000003, 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19697366399999999, 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085798+0000", 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085798+0000", 2024-09-15T01:33:27.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085799+0000", 2024-09-15T01:33:27.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9199999999999999e-07 2024-09-15T01:33:27.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085804+0000", 2024-09-15T01:33:27.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0440000000000003e-06 2024-09-15T01:33:27.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282638+0000", 2024-09-15T01:33:27.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19683383099999999 2024-09-15T01:33:27.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282645+0000", 2024-09-15T01:33:27.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9800000000000001e-06 2024-09-15T01:33:27.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282743+0000", 2024-09-15T01:33:27.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8517999999999994e-05 2024-09-15T01:33:27.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282772+0000", 2024-09-15T01:33:27.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8399e-05 2024-09-15T01:33:27.485 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.485 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.485 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.485 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.485 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.485 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:27.487 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:27.485+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:27.488 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:33:27.488 INFO:tasks.rados.rados.0.smithi175.stdout:322: finishing rollback tid 0 to smithi17529310-13 2024-09-15T01:33:27.488 INFO:tasks.rados.rados.0.smithi175.stdout:322: finishing rollback tid 1 to smithi17529310-13 2024-09-15T01:33:27.488 INFO:tasks.rados.rados.0.smithi175.stdout:322: finishing rollback tid 2 to smithi17529310-13 2024-09-15T01:33:27.488 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 13 v 109 (ObjNum 95 snap 9 seq_num 95) dirty exists 2024-09-15T01:33:27.489 INFO:tasks.rados.rados.0.smithi175.stdout:321: finishing copy_from racing read to smithi17529310-21 2024-09-15T01:33:27.489 INFO:tasks.rados.rados.0.smithi175.stdout:321: finishing copy_from to smithi17529310-21 2024-09-15T01:33:27.489 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 21 v 45 (ObjNum 133 snap 24 seq_num 133) dirty exists 2024-09-15T01:33:27.588 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:27.586+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:27.625 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:27.626 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:27.634 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:33:27.658 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-15T01:33:27.689 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:27.687+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:27.732 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:27.732 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:27.732 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:27.732 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:27.732 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:27.740 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:33:27.790 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:27.788+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:27.838 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:27.838 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:27.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:27.838 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:27.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.838 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:108 3.ds0 3:b063f44c:::smithi17529310-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:27.838 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:27.838 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.588676833000001, 2024-09-15T01:33:27.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013756292, 2024-09-15T01:33:27.838 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 108 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:27.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246459+0000", 2024-09-15T01:33:27.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6800000000000001e-07 2024-09-15T01:33:27.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246461+0000", 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.131e-06 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246468+0000", 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5529999999999998e-06 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246475+0000", 2024-09-15T01:33:27.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5830000000000001e-06 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.248249+0000", 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017739990000000001 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.248333+0000", 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4288000000000003e-05 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.249154+0000", 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00082014900000000005 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.249218+0000", 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4519999999999999e-05 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.260215+0000", 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010996601 2024-09-15T01:33:27.843 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.593912330000002, 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014051065, 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241243+0000", 2024-09-15T01:33:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0126e-05 2024-09-15T01:33:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241422+0000", 2024-09-15T01:33:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017906899999999999 2024-09-15T01:33:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241429+0000", 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4139999999999999e-06 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241500+0000", 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1098000000000003e-05 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241510+0000", 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8470000000000008e-06 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241787+0000", 2024-09-15T01:33:27.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027729700000000003 2024-09-15T01:33:27.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242007+0000", 2024-09-15T01:33:27.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000220134 2024-09-15T01:33:27.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242146+0000", 2024-09-15T01:33:27.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013846799999999999 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242278+0000", 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0001326 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.255274+0000", 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01299575 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.849 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s5 44/38 ECSubWrite(tid=20, reqid=osd.0.0:20, at_version=44'12, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.243644+0000", 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.591491591, 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012378171, 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243644+0000", 2024-09-15T01:33:27.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243644+0000", 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243644+0000", 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243670+0000", 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6384999999999999e-05 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243811+0000", 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014044900000000001 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243817+0000", 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6680000000000004e-06 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244023+0000", 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020566099999999999 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244033+0000", 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9510000000000001e-06 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245269+0000", 2024-09-15T01:33:27.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001236109 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245301+0000", 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1911e-05 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245535+0000", 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023357599999999999 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245580+0000", 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5327999999999997e-05 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.256022+0000", 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010441884 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:27.855 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.591453985000001, 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014188612, 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243682+0000", 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0299999999999999e-07 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243699+0000", 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7575999999999999e-05 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244099+0000", 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039909500000000003 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244108+0000", 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3209999999999994e-06 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244168+0000", 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9688999999999997e-05 2024-09-15T01:33:27.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244180+0000", 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2031000000000001e-05 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245504+0000", 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013241520000000001 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245537+0000", 2024-09-15T01:33:27.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3287000000000002e-05 2024-09-15T01:33:27.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245645+0000", 2024-09-15T01:33:27.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108435 2024-09-15T01:33:27.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245664+0000", 2024-09-15T01:33:27.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8329999999999999e-05 2024-09-15T01:33:27.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.257870+0000", 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012206092999999999 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.588170402000003, 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013797696999999999, 2024-09-15T01:33:27.861 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246966+0000", 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000002e-07 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246979+0000", 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3111e-05 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247017+0000", 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8096999999999998e-05 2024-09-15T01:33:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247025+0000", 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9100000000000005e-06 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247095+0000", 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0049000000000006e-05 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247104+0000", 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7919999999999998e-06 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247217+0000", 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011299 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247376+0000", 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015968800000000001 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247417+0000", 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0228999999999997e-05 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248151+0000", 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00073475999999999999 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.260763+0000", 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012611282999999999 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.866 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.as1 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.247767+0000", 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.587368556000001, 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01354909, 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247767+0000", 2024-09-15T01:33:27.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247767+0000", 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247767+0000", 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4299999999999999e-07 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247778+0000", 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1027e-05 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247838+0000", 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9571000000000003e-05 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247842+0000", 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9740000000000004e-06 2024-09-15T01:33:27.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247901+0000", 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9694999999999997e-05 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247923+0000", 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1767e-05 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248086+0000", 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016259900000000001 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248105+0000", 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9609000000000001e-05 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248157+0000", 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1613999999999997e-05 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248189+0000", 2024-09-15T01:33:27.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2140000000000001e-05 2024-09-15T01:33:27.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.261316+0000", 2024-09-15T01:33:27.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013126851 2024-09-15T01:33:27.872 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.872 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s5 44/38 ECSubWrite(tid=21, reqid=osd.0.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.248138+0000", 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.586997725000003, 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013472511, 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248138+0000", 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248138+0000", 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248138+0000", 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4600000000000001e-07 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248143+0000", 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6489999999999996e-06 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248368+0000", 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022486699999999999 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248375+0000", 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0299999999999996e-06 2024-09-15T01:33:27.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248429+0000", 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3236999999999999e-05 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248438+0000", 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8060000000000006e-06 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248547+0000", 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108527 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248566+0000", 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9525999999999999e-05 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248606+0000", 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9730000000000001e-05 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248621+0000", 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4435e-05 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.261610+0000", 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012989458000000001 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 44/31 ECSubWrite(tid=153, reqid=osd.13.0:152, at_version=0'0, trim_to=0'0, roll_forward_to=44'17))", 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.248843+0000", 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.586291862000003, 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013295741, 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248843+0000", 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248843+0000", 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248844+0000", 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9599999999999999e-07 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248853+0000", 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.755e-06 2024-09-15T01:33:27.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248917+0000", 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3998999999999995e-05 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248923+0000", 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9139999999999996e-06 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248976+0000", 2024-09-15T01:33:27.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3325000000000002e-05 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248986+0000", 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9799999999999993e-06 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.249081+0000", 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5148000000000004e-05 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.249121+0000", 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0111000000000003e-05 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.249145+0000", 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4247999999999999e-05 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.249158+0000", 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3063e-05 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.262139+0000", 2024-09-15T01:33:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012980702 2024-09-15T01:33:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:283 3.ds0 3:bfa066a3:::smithi17529310-39:head [copy-from ver 2] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.576369663999998, 2024-09-15T01:33:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015911307999999999, 2024-09-15T01:33:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 283 2024-09-15T01:33:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258768+0000", 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0980000000000001e-06 2024-09-15T01:33:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258779+0000", 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1272e-05 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258795+0000", 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5679999999999999e-05 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258802+0000", 2024-09-15T01:33:27.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9029999999999998e-06 2024-09-15T01:33:27.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258879+0000", 2024-09-15T01:33:27.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7035999999999999e-05 2024-09-15T01:33:27.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:27.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.259139+0000", 2024-09-15T01:33:27.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260456 2024-09-15T01:33:27.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268247+0000", 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0091076109999999998 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268518+0000", 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027146499999999998 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268542+0000", 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3601e-05 2024-09-15T01:33:27.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268616+0000", 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3944999999999998e-05 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.269087+0000", 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047095399999999999 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:27.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.274577+0000", 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0054903019999999999 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.274677+0000", 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9984999999999999e-05 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 44/31 ECSubWriteReply(tid=43, last_complete=44'32, committed=1, applied=1))", 2024-09-15T01:33:27.891 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.559688798000003, 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.192594981, 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:27.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275447+0000", 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2099999999999995e-07 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275452+0000", 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2050000000000004e-06 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.467889+0000", 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19243722999999999 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.467922+0000", 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3237e-05 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.468017+0000", 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4543999999999995e-05 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.468041+0000", 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4844000000000001e-05 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.5s4 55/31 ECSubWrite(tid=49, reqid=client.4477.0:505, at_version=55'33, trim_to=0'0, roll_forward_to=54'31))", 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.082609+0000", 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 30.752526225, 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012942433999999999, 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.895 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082609+0000", 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082609+0000", 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082609+0000", 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.17e-07 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082659+0000", 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0194000000000001e-05 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082708+0000", 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8046e-05 2024-09-15T01:33:27.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082716+0000", 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7269999999999996e-06 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082772+0000", 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5506e-05 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082781+0000", 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3549999999999994e-06 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083044+0000", 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00026238900000000002 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083106+0000", 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2136000000000003e-05 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083184+0000", 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8133000000000006e-05 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083201+0000", 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6884999999999998e-05 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.095552+0000", 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012350846 2024-09-15T01:33:27.900 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 66/31 ECSubWrite(tid=241, reqid=osd.13.0:241, at_version=66'51, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081422+0000", 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.753713648000002, 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013340902, 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.901 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081422+0000", 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081422+0000", 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081422+0000", 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3300000000000001e-07 2024-09-15T01:33:27.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081438+0000", 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5982999999999999e-05 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081495+0000", 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7491999999999998e-05 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081502+0000", 2024-09-15T01:33:27.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1450000000000002e-06 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081551+0000", 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8640000000000002e-05 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081560+0000", 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2410000000000001e-06 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081748+0000", 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018781399999999999 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083535+0000", 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017866570000000001 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083587+0000", 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1956000000000002e-05 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083618+0000", 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0858999999999997e-05 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.094763+0000", 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011144882 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.906 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 66/31 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=66'53, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.752453573, 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014557652000000001, 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082683+0000", 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2999999999999999e-07 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082704+0000", 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1381000000000001e-05 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082736+0000", 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2239000000000001e-05 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082743+0000", 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6449999999999999e-06 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082927+0000", 2024-09-15T01:33:27.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018380800000000001 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082934+0000", 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1600000000000001e-06 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083133+0000", 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019963000000000001 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083536+0000", 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040219000000000002 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083587+0000", 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1600000000000001e-05 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083644+0000", 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6753999999999999e-05 2024-09-15T01:33:27.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097239+0000", 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013595415 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.5s4 66/31 ECSubWrite(tid=54, reqid=osd.15.0:54, at_version=66'39, trim_to=0'0, roll_forward_to=56'37))", 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.084027+0000", 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.751108274, 2024-09-15T01:33:27.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012243765, 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084027+0000", 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084027+0000", 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084027+0000", 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.99e-07 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084051+0000", 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3657999999999999e-05 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084120+0000", 2024-09-15T01:33:27.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8998999999999994e-05 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084128+0000", 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9619999999999993e-06 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084661+0000", 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000532866 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084670+0000", 2024-09-15T01:33:27.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4939999999999992e-06 2024-09-15T01:33:27.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084860+0000", 2024-09-15T01:33:27.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000189543 2024-09-15T01:33:27.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085085+0000", 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022555399999999999 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085119+0000", 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4152000000000001e-05 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085139+0000", 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9938000000000001e-05 2024-09-15T01:33:27.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.096271+0000", 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0111314 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:609 3.ds0 3:b50d79c0:::smithi17529310-18:head [append 0~770048] snapc 1f=[1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.085186+0000", 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.749948824000001, 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013710833, 2024-09-15T01:33:27.918 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 609 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085186+0000", 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085186+0000", 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085188+0000", 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.694e-06 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085539+0000", 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035108399999999999 2024-09-15T01:33:27.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085555+0000", 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6127999999999999e-05 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085565+0000", 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6819999999999998e-06 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085613+0000", 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7478999999999997e-05 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085720+0000", 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010775799999999999 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087279+0000", 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001558729 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.092083+0000", 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.004803501 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.092092+0000", 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3389999999999999e-06 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.094255+0000", 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021635970000000002 2024-09-15T01:33:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.096016+0000", 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017606130000000001 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.096085+0000", 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9275000000000003e-05 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.098897+0000", 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028119540000000002 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s5 71/69 ECSubWrite(tid=73, reqid=client.4477.0:701, at_version=71'45, trim_to=0'0, roll_forward_to=70'43))", 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.321241+0000", 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.513893836999999, 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014395221999999999, 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321241+0000", 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321241+0000", 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321242+0000", 2024-09-15T01:33:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.36e-07 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327486+0000", 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0062439130000000002 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327562+0000", 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6878999999999994e-05 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327570+0000", 2024-09-15T01:33:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3520000000000001e-06 2024-09-15T01:33:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327617+0000", 2024-09-15T01:33:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6839999999999999e-05 2024-09-15T01:33:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327627+0000", 2024-09-15T01:33:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.927e-06 2024-09-15T01:33:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328144+0000", 2024-09-15T01:33:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051762900000000003 2024-09-15T01:33:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328237+0000", 2024-09-15T01:33:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3153999999999996e-05 2024-09-15T01:33:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328549+0000", 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000311873 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328572+0000", 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2751999999999998e-05 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.335637+0000", 2024-09-15T01:33:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0070647669999999996 2024-09-15T01:33:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.5s4 71/31 ECSubWrite(tid=66, reqid=client.4477.0:714, at_version=71'48, trim_to=0'0, roll_forward_to=71'46))", 2024-09-15T01:33:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.402361+0000", 2024-09-15T01:33:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.432773918000001, 2024-09-15T01:33:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011069041, 2024-09-15T01:33:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402361+0000", 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402361+0000", 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402363+0000", 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3489999999999999e-06 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402386+0000", 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3577000000000001e-05 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402425+0000", 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9122000000000003e-05 2024-09-15T01:33:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402434+0000", 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7099999999999996e-06 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402479+0000", 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4648999999999999e-05 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402489+0000", 2024-09-15T01:33:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9480000000000003e-06 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402688+0000", 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019906200000000001 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402694+0000", 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6100000000000002e-06 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402731+0000", 2024-09-15T01:33:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6328999999999997e-05 2024-09-15T01:33:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402750+0000", 2024-09-15T01:33:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9697000000000001e-05 2024-09-15T01:33:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.413430+0000", 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010679988 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:715 3.ds0 3:b063f44c:::smithi17529310-36:head [copy-from ver 49] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.446209+0000", 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.388925945, 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011676598, 2024-09-15T01:33:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 715 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446209+0000", 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446209+0000", 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446211+0000", 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5659999999999999e-06 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446218+0000", 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8029999999999999e-06 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446244+0000", 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6293999999999999e-05 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446253+0000", 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1279999999999997e-06 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446335+0000", 2024-09-15T01:33:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1366000000000007e-05 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446393+0000", 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8118000000000002e-05 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.454804+0000", 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.008411623 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.455051+0000", 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024685199999999999 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.455103+0000", 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1783000000000002e-05 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.455259+0000", 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000155673 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.455379+0000", 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000120079 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.456386+0000", 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001007228 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.457886+0000", 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001500085 2024-09-15T01:33:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:823 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [copy-from ver 45] snapc 30=[30,2e,2c,2b,29,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:27.126788+0000", 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.70834753500000003, 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.02620683, 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 823 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126788+0000", 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126788+0000", 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126790+0000", 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9360000000000002e-06 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126797+0000", 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0330000000000003e-06 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126824+0000", 2024-09-15T01:33:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7637e-05 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126834+0000", 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.268e-06 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126932+0000", 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7882999999999998e-05 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127109+0000", 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017735299999999999 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.142551+0000", 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015441939 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143350+0000", 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00079953299999999995 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143360+0000", 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0098999999999999e-05 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143663+0000", 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030244499999999998 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143835+0000", 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017216200000000001 2024-09-15T01:33:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.152928+0000", 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0090929219999999998 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.152995+0000", 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6619999999999996e-05 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:824 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.708289481, 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016980479, 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 824 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6199999999999998e-07 2024-09-15T01:33:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126849+0000", 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6800000000000002e-06 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127309+0000", 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045968300000000001 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127316+0000", 2024-09-15T01:33:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9500000000000004e-06 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127358+0000", 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1990999999999998e-05 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127413+0000", 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5448999999999997e-05 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143574+0000", 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016161045999999998 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143678+0000", 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103719 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143826+0000", 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014839900000000001 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:27.956 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:27.958 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-15T01:33:27.959 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:33:27.959 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:27.956+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:27.959 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:27.959 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:28.058 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:28.056+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:28.095 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:28.096 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:28.096 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:28.096 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:28.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:33:28.108 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-15T01:33:28.110 INFO:tasks.rados.rados.0.smithi175.stdout:318: done (5 left) 2024-09-15T01:33:28.110 INFO:tasks.rados.rados.0.smithi175.stdout:319: done (4 left) 2024-09-15T01:33:28.110 INFO:tasks.rados.rados.0.smithi175.stdout:320: done (3 left) 2024-09-15T01:33:28.110 INFO:tasks.rados.rados.0.smithi175.stdout:321: done (2 left) 2024-09-15T01:33:28.110 INFO:tasks.rados.rados.0.smithi175.stdout:322: done (1 left) 2024-09-15T01:33:28.110 INFO:tasks.rados.rados.0.smithi175.stdout:323: done (0 left) 2024-09-15T01:33:28.110 INFO:tasks.rados.rados.0.smithi175.stdout:324: rollback oid 10 current snap is 29 2024-09-15T01:33:28.110 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 10 to 28 2024-09-15T01:33:28.111 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 39 current snap is 29 2024-09-15T01:33:28.111 INFO:tasks.rados.rados.0.smithi175.stdout:325: seq_num 139 ranges {196608=786432,983040=688128,1671168=475136,2146304=393216} 2024-09-15T01:33:28.115 INFO:tasks.rados.rados.0.smithi175.stdout:325: writing smithi17529310-39 from 196608 to 983040 tid 1 2024-09-15T01:33:28.120 INFO:tasks.rados.rados.0.smithi175.stdout:325: writing smithi17529310-39 from 983040 to 1671168 tid 2 2024-09-15T01:33:28.123 INFO:tasks.rados.rados.0.smithi175.stdout:325: writing smithi17529310-39 from 1671168 to 2146304 tid 3 2024-09-15T01:33:28.126 INFO:tasks.rados.rados.0.smithi175.stdout:325: writing smithi17529310-39 from 2146304 to 2539520 tid 4 2024-09-15T01:33:28.126 INFO:tasks.rados.rados.0.smithi175.stdout:326: rollback oid 40 current snap is 29 2024-09-15T01:33:28.126 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 40 to 28 2024-09-15T01:33:28.126 INFO:tasks.rados.rados.0.smithi175.stdout:324: finishing rollback tid 0 to smithi17529310-10 2024-09-15T01:33:28.127 INFO:tasks.rados.rados.0.smithi175.stdout:324: finishing rollback tid 1 to smithi17529310-10 2024-09-15T01:33:28.127 INFO:tasks.rados.rados.0.smithi175.stdout:324: finishing rollback tid 2 to smithi17529310-10 2024-09-15T01:33:28.127 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 10 v 91 (ObjNum 58 snap 3 seq_num 58) dirty exists 2024-09-15T01:33:28.127 INFO:tasks.rados.rados.0.smithi175.stdout:325: finishing write tid 1 to smithi17529310-39 2024-09-15T01:33:28.127 INFO:tasks.rados.rados.0.smithi175.stdout:325: finishing write tid 2 to smithi17529310-39 2024-09-15T01:33:28.127 INFO:tasks.rados.rados.0.smithi175.stdout:324: done (2 left) 2024-09-15T01:33:28.127 INFO:tasks.rados.rados.0.smithi175.stdout:327: read oid 36 snap -1 2024-09-15T01:33:28.127 INFO:tasks.rados.rados.0.smithi175.stdout:327: expect (ObjNum 131 snap 21 seq_num 131) 2024-09-15T01:33:28.127 INFO:tasks.rados.rados.0.smithi175.stdout:328: delete oid 38 current snap is 29 2024-09-15T01:33:28.130 INFO:tasks.rados.rados.0.smithi175.stdout:326: finishing rollback tid 0 to smithi17529310-40 2024-09-15T01:33:28.130 INFO:tasks.rados.rados.0.smithi175.stdout:328: done (3 left) 2024-09-15T01:33:28.130 INFO:tasks.rados.rados.0.smithi175.stdout:329: delete oid 50 current snap is 29 2024-09-15T01:33:28.130 INFO:tasks.rados.rados.0.smithi175.stdout:326: finishing rollback tid 1 to smithi17529310-40 2024-09-15T01:33:28.132 INFO:tasks.rados.rados.0.smithi175.stdout:326: finishing rollback tid 2 to smithi17529310-40 2024-09-15T01:33:28.132 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 40 v 45 (ObjNum 74 snap 5 seq_num 74) dirty exists 2024-09-15T01:33:28.133 INFO:tasks.rados.rados.0.smithi175.stdout:326: done (3 left) 2024-09-15T01:33:28.133 INFO:tasks.rados.rados.0.smithi175.stdout:329: done (2 left) 2024-09-15T01:33:28.133 INFO:tasks.rados.rados.0.smithi175.stdout:330: rollback oid 45 current snap is 29 2024-09-15T01:33:28.133 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 45 to 25 2024-09-15T01:33:28.133 INFO:tasks.rados.rados.0.smithi175.stdout:331: snap_remove snap 13 2024-09-15T01:33:28.149 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-15T01:33:28.158 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:28.157+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:28.210 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:28.210 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:28.210 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:28.211 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:28.211 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:28.218 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:33:28.260 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:28.258+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:28.332 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-15T01:33:28.357 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:28.357 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:99 3.fs0 3:f630fe6e:::smithi17529310-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.119886729000001, 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055995037999999997, 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99 2024-09-15T01:33:28.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235203+0000", 2024-09-15T01:33:28.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4000000000000002e-07 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235205+0000", 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3e-06 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235212+0000", 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4389999999999997e-06 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235216+0000", 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.031e-06 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.236877+0000", 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00166088 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.236933+0000", 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6873999999999998e-05 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255906+0000", 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018972066999999999 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255957+0000", 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1467e-05 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291198+0000", 2024-09-15T01:33:28.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035240540000000001 2024-09-15T01:33:28.363 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.363 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.363 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.363 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:114 3.fs0 3:f8072b57:::smithi17529310-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:28.363 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:28.363 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.101446807999999, 2024-09-15T01:33:28.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048699039999999999, 2024-09-15T01:33:28.363 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.364 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:28.364 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.364 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.364 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.364 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 114 2024-09-15T01:33:28.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.364 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:28.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:28.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253643+0000", 2024-09-15T01:33:28.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6899999999999998e-07 2024-09-15T01:33:28.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253646+0000", 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253652+0000", 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3369999999999996e-06 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253656+0000", 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.031e-06 2024-09-15T01:33:28.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255410+0000", 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017543420000000001 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255505+0000", 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4704999999999995e-05 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290833+0000", 2024-09-15T01:33:28.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035327711999999997 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290893+0000", 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0384999999999999e-05 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.302341+0000", 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011448274 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.369 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:145 3.cs0 3:34a45b5f:::smithi17529310-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:28.369 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:28.369 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.057220239999999, 2024-09-15T01:33:28.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15351202899999999, 2024-09-15T01:33:28.369 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.369 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:28.369 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.369 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.369 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.369 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 145 2024-09-15T01:33:28.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.370 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:28.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:28.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:28.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0500000000000004e-07 2024-09-15T01:33:28.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297871+0000", 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9769999999999999e-06 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297878+0000", 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9120000000000001e-06 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297882+0000", 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0910000000000003e-06 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.298932+0000", 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00104904 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.299009+0000", 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7095000000000006e-05 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.306936+0000", 2024-09-15T01:33:28.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0079278730000000002 2024-09-15T01:33:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.307022+0000", 2024-09-15T01:33:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5254999999999996e-05 2024-09-15T01:33:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.451381+0000", 2024-09-15T01:33:28.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14435928100000001 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:312 3.cs0 3:37dffec7:::smithi17529310-13:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 42.076896625000003, 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770305000000002, 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.375 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 312 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0200000000000002e-07 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278196+0000", 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7319999999999998e-06 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278216+0000", 2024-09-15T01:33:28.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0375999999999999e-05 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278221+0000", 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4889999999999997e-06 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279668+0000", 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014473929999999999 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279753+0000", 2024-09-15T01:33:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4567999999999994e-05 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280213+0000", 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00046030399999999998 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280327+0000", 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011404399999999999 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280342+0000", 2024-09-15T01:33:28.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4403000000000001e-05 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280467+0000", 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000125594 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280562+0000", 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5317999999999998e-05 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:28.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328870+0000", 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048307782 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328963+0000", 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2700000000000004e-05 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:313 3.cs0 3:37dffec7:::smithi17529310-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:28.381 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 42.076863744000001, 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053267040000000002, 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 313 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278226+0000", 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7100000000000002e-07 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278228+0000", 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1859999999999999e-06 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278235+0000", 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9020000000000002e-06 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278240+0000", 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9540000000000003e-06 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280404+0000", 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021641469999999999 2024-09-15T01:33:28.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280503+0000", 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9090000000000002e-05 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328996+0000", 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048492533999999997 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.329053+0000", 2024-09-15T01:33:28.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6997999999999999e-05 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.331493+0000", 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024397580000000002 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=95, last_complete=44'34, committed=1, applied=1))", 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 42.073459401000001, 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047344538999999998, 2024-09-15T01:33:28.386 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7700000000000001e-07 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281632+0000", 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.562e-06 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328775+0000", 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047143813 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328779+0000", 2024-09-15T01:33:28.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.895e-06 2024-09-15T01:33:28.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328803+0000", 2024-09-15T01:33:28.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4076000000000001e-05 2024-09-15T01:33:28.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328974+0000", 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017101599999999999 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:324 3.cs0 3:34a45b5f:::smithi17529310-48:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 42.024444293999998, 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18862163400000001, 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:28.390 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 324 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:28.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699999999999999e-07 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330648+0000", 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.694e-06 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330654+0000", 2024-09-15T01:33:28.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4090000000000002e-06 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330659+0000", 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3210000000000004e-06 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332198+0000", 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015388699999999999 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332262+0000", 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4196000000000007e-05 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332785+0000", 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052292099999999998 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332870+0000", 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5049999999999994e-05 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332877+0000", 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7769999999999997e-06 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332902+0000", 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5012e-05 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332962+0000", 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0649999999999997e-05 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519220+0000", 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18625739199999999 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519267+0000", 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6875000000000001e-05 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:325 3.cs0 3:34a45b5f:::smithi17529310-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 42.024426231, 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.189890015, 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 325 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330667+0000", 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4410000000000002e-06 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330673+0000", 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5429999999999999e-06 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330676+0000", 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.683e-06 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333439+0000", 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002763415 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333492+0000", 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2760000000000003e-05 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519496+0000", 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18600408199999999 2024-09-15T01:33:28.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519560+0000", 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3660999999999999e-05 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.520553+0000", 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00099301399999999996 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=100, last_complete=44'39, committed=1, applied=1))", 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 42.022024188000003, 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18640773799999999, 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5999999999999999e-08 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333066+0000", 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.043e-06 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519088+0000", 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18602205399999999 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519092+0000", 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0890000000000002e-06 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519171+0000", 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8857999999999996e-05 2024-09-15T01:33:28.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519473+0000", 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030159799999999998 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 47/31 ECSubWriteReply(tid=113, last_complete=47'51, committed=1, applied=1))", 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 39.095202377, 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.63648575600000001, 2024-09-15T01:33:28.405 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-07 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259890+0000", 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5859999999999999e-06 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.895924+0000", 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.63603432699999995 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.895929+0000", 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1340000000000003e-06 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.896000+0000", 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0687999999999998e-05 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.896373+0000", 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037265200000000001 2024-09-15T01:33:28.408 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 48/31 ECSubWriteReply(tid=128, last_complete=48'57, committed=1, applied=1))", 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 38.048386888000003, 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18111660500000001, 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306703+0000", 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3e-07 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306707+0000", 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7500000000000001e-06 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487741+0000", 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18103418199999999 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487745+0000", 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2189999999999998e-06 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487789+0000", 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3430000000000003e-05 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487819+0000", 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0394000000000001e-05 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 53/31 ECSubWriteReply(tid=146, last_complete=53'74, committed=1, applied=1))", 2024-09-15T01:33:28.412 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.154545439000003, 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.74248160299999999, 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-07 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200550+0000", 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5910000000000003e-06 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.942942+0000", 2024-09-15T01:33:28.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.74239190499999996 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.942965+0000", 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2983999999999999e-05 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.943000+0000", 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5336999999999997e-05 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.943025+0000", 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5250999999999999e-05 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.7s6 55/52 [PushOp(3:e6e8875f:::smithi17529310-31:head, version: 50'31, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:e6e8875f:::smithi17529310-31:head@50'31, size: 1949696, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.256830833999999, 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18071695199999999, 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098259+0000", 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0200000000000001e-07 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098545+0000", 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028574699999999999 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278693+0000", 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.180148009 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278700+0000", 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.888e-06 2024-09-15T01:33:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278803+0000", 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102766 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278975+0000", 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017284000000000001 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:524 3.fs0 3:f8e751c8:::smithi17529310-17:head [setxattr _header (57) in=7b] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 30.261066361000001, 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.115556992, 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 524 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2200000000000004e-07 2024-09-15T01:33:28.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094028+0000", 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9710000000000003e-06 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094040+0000", 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1066e-05 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094044+0000", 2024-09-15T01:33:28.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8770000000000001e-06 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095122+0000", 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010771839999999999 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095188+0000", 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6298999999999997e-05 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095521+0000", 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033339600000000002 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095614+0000", 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2295999999999999e-05 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095620+0000", 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8000000000000001e-06 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095653+0000", 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2867000000000002e-05 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095740+0000", 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6300999999999999e-05 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209497+0000", 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.113757184 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209580+0000", 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3128999999999996e-05 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:525 3.fs0 3:f8e751c8:::smithi17529310-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 30.261040535999999, 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11664384, 2024-09-15T01:33:28.426 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 525 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:28.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:28.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6100000000000001e-07 2024-09-15T01:33:28.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094051+0000", 2024-09-15T01:33:28.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1950000000000002e-06 2024-09-15T01:33:28.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094058+0000", 2024-09-15T01:33:28.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4350000000000002e-06 2024-09-15T01:33:28.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094063+0000", 2024-09-15T01:33:28.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2039999999999996e-06 2024-09-15T01:33:28.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095664+0000", 2024-09-15T01:33:28.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001600642 2024-09-15T01:33:28.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:28.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095725+0000", 2024-09-15T01:33:28.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1707999999999999e-05 2024-09-15T01:33:28.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209923+0000", 2024-09-15T01:33:28.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.114197744 2024-09-15T01:33:28.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209997+0000", 2024-09-15T01:33:28.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3341000000000001e-05 2024-09-15T01:33:28.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.210693+0000", 2024-09-15T01:33:28.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00069601000000000005 2024-09-15T01:33:28.435 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.435 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.435 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.436 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:691 3.fs0 3:f73c1a72:::smithi17529310-29:head [copy-from ver 48] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:28.436 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.296702+0000", 2024-09-15T01:33:28.436 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.058386958, 2024-09-15T01:33:28.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.042728760999999997, 2024-09-15T01:33:28.436 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.437 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:28.437 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.437 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.437 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.437 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 691 2024-09-15T01:33:28.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.438 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296702+0000", 2024-09-15T01:33:28.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296702+0000", 2024-09-15T01:33:28.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296703+0000", 2024-09-15T01:33:28.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-15T01:33:28.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296704+0000", 2024-09-15T01:33:28.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7039999999999999e-06 2024-09-15T01:33:28.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296708+0000", 2024-09-15T01:33:28.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0439999999999997e-06 2024-09-15T01:33:28.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296710+0000", 2024-09-15T01:33:28.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0200000000000001e-06 2024-09-15T01:33:28.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.299482+0000", 2024-09-15T01:33:28.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027719820000000001 2024-09-15T01:33:28.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.299578+0000", 2024-09-15T01:33:28.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5361999999999999e-05 2024-09-15T01:33:28.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321672+0000", 2024-09-15T01:33:28.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022094721000000001 2024-09-15T01:33:28.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322236+0000", 2024-09-15T01:33:28.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00056322499999999999 2024-09-15T01:33:28.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322250+0000", 2024-09-15T01:33:28.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.438e-05 2024-09-15T01:33:28.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322630+0000", 2024-09-15T01:33:28.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00038026599999999998 2024-09-15T01:33:28.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322764+0000", 2024-09-15T01:33:28.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013368899999999999 2024-09-15T01:33:28.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:28.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.339358+0000", 2024-09-15T01:33:28.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016593988 2024-09-15T01:33:28.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.339431+0000", 2024-09-15T01:33:28.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3041999999999996e-05 2024-09-15T01:33:28.451 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.451 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.451 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.452 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:746 3.9s0 3:98b2f2fd:::smithi17529310-14:head [append 0~0] snapc 27=[27,26,24,22,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-15T01:33:28.452 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:18.078177+0000", 2024-09-15T01:33:28.452 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.276912081000001, 2024-09-15T01:33:28.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0091383140000001, 2024-09-15T01:33:28.452 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.453 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:28.453 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.453 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.453 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.453 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 746 2024-09-15T01:33:28.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.454 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078177+0000", 2024-09-15T01:33:28.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078177+0000", 2024-09-15T01:33:28.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078178+0000", 2024-09-15T01:33:28.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1799999999999995e-07 2024-09-15T01:33:28.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078181+0000", 2024-09-15T01:33:28.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.038e-06 2024-09-15T01:33:28.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078187+0000", 2024-09-15T01:33:28.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4760000000000003e-06 2024-09-15T01:33:28.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078191+0000", 2024-09-15T01:33:28.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7189999999999999e-06 2024-09-15T01:33:28.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078448+0000", 2024-09-15T01:33:28.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025739300000000002 2024-09-15T01:33:28.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op must wait for map", 2024-09-15T01:33:28.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078457+0000", 2024-09-15T01:33:28.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7590000000000003e-06 2024-09-15T01:33:28.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.082114+0000", 2024-09-15T01:33:28.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036565579999999999 2024-09-15T01:33:28.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-15T01:33:28.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.082117+0000", 2024-09-15T01:33:28.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0299999999999998e-06 2024-09-15T01:33:28.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.084154+0000", 2024-09-15T01:33:28.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0020370750000001 2024-09-15T01:33:28.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.084312+0000", 2024-09-15T01:33:28.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015769300000000001 2024-09-15T01:33:28.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.084835+0000", 2024-09-15T01:33:28.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052329899999999996 2024-09-15T01:33:28.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.085132+0000", 2024-09-15T01:33:28.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029761499999999999 2024-09-15T01:33:28.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.085143+0000", 2024-09-15T01:33:28.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0951000000000001e-05 2024-09-15T01:33:28.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.085193+0000", 2024-09-15T01:33:28.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9369000000000001e-05 2024-09-15T01:33:28.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.086029+0000", 2024-09-15T01:33:28.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00083588200000000003 2024-09-15T01:33:28.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:28.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.086781+0000", 2024-09-15T01:33:28.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00075188200000000005 2024-09-15T01:33:28.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087315+0000", 2024-09-15T01:33:28.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00053495699999999997 2024-09-15T01:33:28.471 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.471 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.471 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.472 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:747 3.9s0 3:98b2f2fd:::smithi17529310-14:head [rollback 19] snapc 27=[27,26,24,22,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-15T01:33:28.472 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:28.472 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.276615687, 2024-09-15T01:33:28.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.011011909, 2024-09-15T01:33:28.473 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.473 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:28.473 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.473 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.473 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.473 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 747 2024-09-15T01:33:28.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.474 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:28.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:28.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:28.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8299999999999995e-07 2024-09-15T01:33:28.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078479+0000", 2024-09-15T01:33:28.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5549999999999996e-06 2024-09-15T01:33:28.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078489+0000", 2024-09-15T01:33:28.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0339999999999999e-05 2024-09-15T01:33:28.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078495+0000", 2024-09-15T01:33:28.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4999999999999999e-06 2024-09-15T01:33:28.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078537+0000", 2024-09-15T01:33:28.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2194999999999998e-05 2024-09-15T01:33:28.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:28.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078544+0000", 2024-09-15T01:33:28.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0759999999999997e-06 2024-09-15T01:33:28.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.084273+0000", 2024-09-15T01:33:28.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057284170000000004 2024-09-15T01:33:28.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-15T01:33:28.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.084275+0000", 2024-09-15T01:33:28.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4049999999999998e-06 2024-09-15T01:33:28.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087195+0000", 2024-09-15T01:33:28.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.002919946 2024-09-15T01:33:28.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087315+0000", 2024-09-15T01:33:28.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000119883 2024-09-15T01:33:28.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087944+0000", 2024-09-15T01:33:28.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00062966699999999999 2024-09-15T01:33:28.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088093+0000", 2024-09-15T01:33:28.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000148531 2024-09-15T01:33:28.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088104+0000", 2024-09-15T01:33:28.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1381e-05 2024-09-15T01:33:28.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088151+0000", 2024-09-15T01:33:28.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6495000000000001e-05 2024-09-15T01:33:28.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088666+0000", 2024-09-15T01:33:28.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051494400000000004 2024-09-15T01:33:28.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:28.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.089420+0000", 2024-09-15T01:33:28.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00075379099999999997 2024-09-15T01:33:28.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.089485+0000", 2024-09-15T01:33:28.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5900000000000003e-05 2024-09-15T01:33:28.491 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.491 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.491 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.492 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 81/76 ECSubWriteReply(tid=313, last_complete=81'63, committed=1, applied=1))", 2024-09-15T01:33:28.492 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:23.083561+0000", 2024-09-15T01:33:28.492 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2715278139999997, 2024-09-15T01:33:28.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87779885599999996, 2024-09-15T01:33:28.492 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.492 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.493 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083561+0000", 2024-09-15T01:33:28.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083561+0000", 2024-09-15T01:33:28.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083562+0000", 2024-09-15T01:33:28.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5400000000000002e-07 2024-09-15T01:33:28.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083564+0000", 2024-09-15T01:33:28.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4739999999999999e-06 2024-09-15T01:33:28.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.960927+0000", 2024-09-15T01:33:28.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87736222600000002 2024-09-15T01:33:28.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.960933+0000", 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3550000000000001e-06 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.961015+0000", 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1646999999999999e-05 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.961360+0000", 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0003457 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.498 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:813 3.cs0 3:3b31c869:::smithi17529310-46:2b [read 0~3981312,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:27.099450+0000", 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.2556395, 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.037884539000000002, 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 813 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.499 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.099450+0000", 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.099450+0000", 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.099450+0000", 2024-09-15T01:33:28.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5199999999999998e-07 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.099452+0000", 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.407e-06 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.099456+0000", 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8700000000000002e-06 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.099461+0000", 2024-09-15T01:33:28.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8810000000000004e-06 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.099530+0000", 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8380999999999995e-05 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.099535+0000", 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7180000000000001e-06 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.102269+0000", 2024-09-15T01:33:28.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027334880000000001 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.102367+0000", 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8517999999999994e-05 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.137334+0000", 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034966924000000003 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.503 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.504 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:28.505 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-15T01:33:28.506 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:28.504+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:28.602 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:28.603 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:28.603 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:28.603 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:28.606 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:28.604+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:28.611 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-15T01:33:28.648 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-15T01:33:28.706 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:28.706 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:28.706 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:28.706 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:28.707 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:28.707 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:28.704+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:28.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-15T01:33:28.807 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:28.804+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:28.816 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:28.816 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:28.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:28.816 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:28.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.816 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:11 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:28.816 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:28.816 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.636472828000002, 2024-09-15T01:33:28.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032033109999999997, 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 11 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176990+0000", 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2200000000000002e-07 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176995+0000", 2024-09-15T01:33:28.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8040000000000001e-06 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177006+0000", 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1276e-05 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177012+0000", 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9370000000000001e-06 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177055+0000", 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2472999999999998e-05 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177197+0000", 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142519 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177567+0000", 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036945000000000001 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177659+0000", 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1928999999999993e-05 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177694+0000", 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4885000000000002e-05 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177722+0000", 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8334e-05 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177733+0000", 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1491000000000001e-05 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208963+0000", 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031230106000000001 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209022+0000", 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8984000000000003e-05 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.822 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:12 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.635722559999998, 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033440417, 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.823 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 12 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2500000000000005e-07 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177745+0000", 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7269999999999998e-06 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177755+0000", 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9000000000000001e-06 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177760+0000", 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1150000000000002e-06 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177801+0000", 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0802e-05 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177877+0000", 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6604e-05 2024-09-15T01:33:28.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209067+0000", 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031189505999999999 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209144+0000", 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6728000000000001e-05 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211180+0000", 2024-09-15T01:33:28.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020363099999999999 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=12, reqid=client.4477.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.623518347000001, 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022201101000000001, 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.75e-07 2024-09-15T01:33:28.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189957+0000", 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2531000000000001e-05 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211941+0000", 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021983546999999999 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211947+0000", 2024-09-15T01:33:28.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7329999999999997e-06 2024-09-15T01:33:28.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211977+0000", 2024-09-15T01:33:28.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9607e-05 2024-09-15T01:33:28.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211982+0000", 2024-09-15T01:33:28.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6290000000000001e-06 2024-09-15T01:33:28.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212053+0000", 2024-09-15T01:33:28.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1890000000000005e-05 2024-09-15T01:33:28.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212060+0000", 2024-09-15T01:33:28.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3339999999999998e-06 2024-09-15T01:33:28.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212087+0000", 2024-09-15T01:33:28.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7314e-05 2024-09-15T01:33:28.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212098+0000", 2024-09-15T01:33:28.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1141999999999999e-05 2024-09-15T01:33:28.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212145+0000", 2024-09-15T01:33:28.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699000000000001e-05 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:72 3.5s0 3:af8bec4d:::smithi17529310-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.606145613000002, 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.037268951000000002, 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.834 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.835 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72 2024-09-15T01:33:28.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.835 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:28.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.34e-07 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207319+0000", 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207326+0000", 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4860000000000002e-06 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207330+0000", 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0690000000000003e-06 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208687+0000", 2024-09-15T01:33:28.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001356966 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208760+0000", 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2663000000000005e-05 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210380+0000", 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001620659 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210464+0000", 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3387999999999995e-05 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.244586+0000", 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034121925999999997 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.0s2 34/31 ECSubWrite(tid=15, reqid=client.4477.0:74, at_version=34'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:33:28.839 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.601034865999999, 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069361051000000007, 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:28.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212428+0000", 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3399999999999999e-07 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212444+0000", 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5681000000000001e-05 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281357+0000", 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068913881999999996 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281366+0000", 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9740000000000008e-06 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281415+0000", 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8498000000000002e-05 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281421+0000", 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4590000000000003e-06 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281487+0000", 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5590000000000001e-05 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281500+0000", 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3298e-05 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281575+0000", 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5247000000000006e-05 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281597+0000", 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1872999999999999e-05 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281788+0000", 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000191015 2024-09-15T01:33:28.844 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 38/31 ECSubWrite(tid=17, reqid=client.4477.0:208, at_version=38'12, trim_to=0'0, roll_forward_to=36'8))", 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 48.592100967999997, 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.93429882900000005, 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:28.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221362+0000", 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9699999999999996e-07 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221380+0000", 2024-09-15T01:33:28.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.804e-05 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.154929+0000", 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.93354887600000003 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.154938+0000", 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6220000000000004e-06 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155018+0000", 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0136000000000007e-05 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155027+0000", 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8259999999999999e-06 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155116+0000", 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8939999999999999e-05 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155127+0000", 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0981e-05 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155185+0000", 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7828999999999998e-05 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155204+0000", 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8649999999999999e-05 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155660+0000", 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000456432 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 51/31 ECSubWrite(tid=56, reqid=osd.5.0:56, at_version=51'43, trim_to=0'0, roll_forward_to=50'41))", 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.736611234000002, 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.063539848999999995, 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.850 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076876+0000", 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4834999999999999e-05 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139053+0000", 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.062177018000000001 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139067+0000", 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3842999999999999e-05 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139206+0000", 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013896500000000001 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139221+0000", 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5193e-05 2024-09-15T01:33:28.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139472+0000", 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025116 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139504+0000", 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1282999999999998e-05 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139541+0000", 2024-09-15T01:33:28.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7518999999999998e-05 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139557+0000", 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5781e-05 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.140391+0000", 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00083395699999999997 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:505 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [copy-from ver 2] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.740423890999999, 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15814687399999999, 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 505 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073040+0000", 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.792e-06 2024-09-15T01:33:28.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073047+0000", 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0169999999999999e-06 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073064+0000", 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.668e-05 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073081+0000", 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7000999999999999e-05 2024-09-15T01:33:28.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073158+0000", 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6779000000000005e-05 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073284+0000", 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000126802 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082517+0000", 2024-09-15T01:33:28.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0092324069999999998 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082822+0000", 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030544099999999999 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082921+0000", 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8760999999999999e-05 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082994+0000", 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2769999999999996e-05 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083217+0000", 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000223706 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231110+0000", 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.147892096 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231185+0000", 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5622000000000002e-05 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 55/31 ECSubWriteReply(tid=49, last_complete=55'33, committed=1, applied=1))", 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.729180578000001, 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14718672399999999, 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9599999999999998e-07 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084287+0000", 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.230928+0000", 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14664121799999999 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.230933+0000", 2024-09-15T01:33:28.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4089999999999996e-06 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231018+0000", 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5589999999999999e-05 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231468+0000", 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045001999999999998 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s4 56/54 ECSubWrite(tid=228, reqid=client.4477.0:520, at_version=56'33, trim_to=0'0, roll_forward_to=55'31))", 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 30.739460547, 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016830581000000001, 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7000000000000003e-07 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074027+0000", 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4845e-05 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074072+0000", 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4583000000000003e-05 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074080+0000", 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6699999999999994e-06 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074280+0000", 2024-09-15T01:33:28.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020015699999999999 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074290+0000", 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0586e-05 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074562+0000", 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027202599999999997 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074573+0000", 2024-09-15T01:33:28.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0974e-05 2024-09-15T01:33:28.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074624+0000", 2024-09-15T01:33:28.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0385e-05 2024-09-15T01:33:28.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074786+0000", 2024-09-15T01:33:28.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016278399999999999 2024-09-15T01:33:28.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.090832+0000", 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016045700999999999 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 66/31 ECSubWrite(tid=130, reqid=osd.14.0:130, at_version=66'56, trim_to=0'0, roll_forward_to=60'52))", 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.733202887000001, 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.31166652299999997, 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.871 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2000000000000001e-07 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080275+0000", 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.556e-05 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.390960+0000", 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.31068479999999998 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.390970+0000", 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0223e-05 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391016+0000", 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5972e-05 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391025+0000", 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8610000000000004e-06 2024-09-15T01:33:28.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391232+0000", 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020709799999999999 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391264+0000", 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2347e-05 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391319+0000", 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5096999999999998e-05 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391336+0000", 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6212e-05 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391926+0000", 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00059013299999999996 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.876 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:646 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [copy-from ver 59] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.077480+0000", 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.735982592999999, 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024635151000000001, 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.877 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 646 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077480+0000", 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077480+0000", 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077482+0000", 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.967e-06 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077494+0000", 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2351000000000001e-05 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077510+0000", 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6461e-05 2024-09-15T01:33:28.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077528+0000", 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7297000000000001e-05 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077583+0000", 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4877999999999999e-05 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op must wait for map", 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077588+0000", 2024-09-15T01:33:28.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5809999999999996e-06 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.078844+0000", 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001255693 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.078981+0000", 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013696499999999999 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097046+0000", 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018065193 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097686+0000", 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00063973600000000002 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097704+0000", 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7934000000000001e-05 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098011+0000", 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030703799999999998 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098161+0000", 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000150233 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.102063+0000", 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003902109 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.102115+0000", 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1715000000000002e-05 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:647 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.077541+0000", 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.735921446000001, 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020616064, 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.884 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 647 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077541+0000", 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077541+0000", 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077542+0000", 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2099999999999995e-07 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077545+0000", 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5360000000000001e-06 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077555+0000", 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0193e-05 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077561+0000", 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5679999999999999e-06 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077630+0000", 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9648999999999997e-05 2024-09-15T01:33:28.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077635+0000", 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0050000000000004e-06 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.079228+0000", 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015930009999999999 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.079271+0000", 2024-09-15T01:33:28.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2588e-05 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097903+0000", 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018632124999999999 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098048+0000", 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000144979 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098157+0000", 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010859899999999999 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:676 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [setxattr _header (56) in=7b] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.542891663000001, 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054773589999999997, 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.890 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 676 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1199999999999999e-07 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270573+0000", 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7409999999999999e-06 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270576+0000", 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0580000000000002e-06 2024-09-15T01:33:28.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270579+0000", 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0630000000000002e-06 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273029+0000", 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024502119999999998 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273139+0000", 2024-09-15T01:33:28.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110196 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273456+0000", 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031676000000000002 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273543+0000", 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7276999999999998e-05 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273550+0000", 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2119999999999997e-06 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273583+0000", 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2625e-05 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273919+0000", 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033556900000000003 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.895 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325294+0000", 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051375250999999997 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325344+0000", 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0314e-05 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:677 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:28.896 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.542870368999999, 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.058358134999999998, 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 677 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.65e-07 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270594+0000", 2024-09-15T01:33:28.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4219999999999999e-06 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270598+0000", 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9439999999999998e-06 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270600+0000", 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.243e-06 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.274286+0000", 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036864089999999999 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.274337+0000", 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0677999999999998e-05 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325802+0000", 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051465576999999998 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325904+0000", 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010192 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328950+0000", 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030455769999999998 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 71/31 ECSubWriteReply(tid=63, last_complete=71'46, committed=1, applied=1))", 2024-09-15T01:33:28.901 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.273846+0000", 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.539615819, 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051825606000000003, 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273846+0000", 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273846+0000", 2024-09-15T01:33:28.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273847+0000", 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5199999999999997e-07 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273850+0000", 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1690000000000001e-06 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325190+0000", 2024-09-15T01:33:28.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051340157999999997 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325198+0000", 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4839999999999998e-06 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325235+0000", 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7156000000000001e-05 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325672+0000", 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00043708700000000001 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 71/31 ECSubWriteReply(tid=64, last_complete=71'46, committed=1, applied=1))", 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.329874+0000", 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.483588189000001, 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.074638840999999997, 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.905 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329874+0000", 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329874+0000", 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329875+0000", 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2799999999999996e-07 2024-09-15T01:33:28.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329878+0000", 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7579999999999998e-06 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404471+0000", 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.074592695000000001 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404475+0000", 2024-09-15T01:33:28.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8650000000000003e-06 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404497+0000", 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2458000000000001e-05 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404513+0000", 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5537e-05 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:714 3.5s0 3:a0a9f8ce:::smithi17529310-35:head [delete] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+full_force+supports_pool_eio e71)", 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.401661+0000", 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.411801338, 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044420634000000001, 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 714 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401661+0000", 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401661+0000", 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401663+0000", 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-15T01:33:28.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401670+0000", 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3549999999999999e-06 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401687+0000", 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7388e-05 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401695+0000", 2024-09-15T01:33:28.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6310000000000003e-06 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401732+0000", 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7415e-05 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401813+0000", 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0117999999999994e-05 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402335+0000", 2024-09-15T01:33:28.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052289800000000005 2024-09-15T01:33:28.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402584+0000", 2024-09-15T01:33:28.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024863700000000001 2024-09-15T01:33:28.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402594+0000", 2024-09-15T01:33:28.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7890000000000006e-06 2024-09-15T01:33:28.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402634+0000", 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9984000000000002e-05 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402647+0000", 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3105000000000001e-05 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:28.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446020+0000", 2024-09-15T01:33:28.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.043373215 2024-09-15T01:33:28.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446081+0000", 2024-09-15T01:33:28.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1248999999999996e-05 2024-09-15T01:33:28.915 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.915 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.915 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.915 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 71/31 ECSubWriteReply(tid=66, last_complete=71'48, committed=1, applied=1))", 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.409007482, 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041833197000000003, 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2099999999999997e-07 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404458+0000", 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.976e-06 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.445921+0000", 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041463284000000003 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.445925+0000", 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0260000000000001e-06 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.445975+0000", 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9424e-05 2024-09-15T01:33:28.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446288+0000", 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031296600000000003 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s4 86/54 ECSubWrite(tid=305, reqid=client.4477.0:848, at_version=86'56, trim_to=0'0, roll_forward_to=85'52))", 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.135651+0000", 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.67781125499999995, 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01654541, 2024-09-15T01:33:28.919 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:28.920 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:28.920 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:28.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:28.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135651+0000", 2024-09-15T01:33:28.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:28.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135651+0000", 2024-09-15T01:33:28.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:28.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:28.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135651+0000", 2024-09-15T01:33:28.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2000000000000001e-07 2024-09-15T01:33:28.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:28.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135665+0000", 2024-09-15T01:33:28.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3978000000000001e-05 2024-09-15T01:33:28.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135726+0000", 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1020000000000002e-05 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135730+0000", 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.929e-06 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135946+0000", 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021585699999999999 2024-09-15T01:33:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135955+0000", 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0929999999999992e-06 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136046+0000", 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0948999999999999e-05 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:28.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136096+0000", 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0049999999999997e-05 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136268+0000", 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000172127 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136324+0000", 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5379e-05 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:28.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:28.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.152196+0000", 2024-09-15T01:33:28.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015872807999999999 2024-09-15T01:33:28.925 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.925 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.925 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.925 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:28.925 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:28.925 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:28.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-15T01:33:28.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:33:28.928 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:28.925+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:29.027 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:29.026+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:29.030 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:29.030 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:29.030 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:33:29.030 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:29.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:33:29.070 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 reset_purged_snaps_last 2024-09-15T01:33:29.119 INFO:tasks.rados.rados.0.smithi175.stdout:331: done (3 left) 2024-09-15T01:33:29.119 INFO:tasks.rados.rados.0.smithi175.stdout:332: read oid 17 snap -1 2024-09-15T01:33:29.119 INFO:tasks.rados.rados.0.smithi175.stdout:332: expect (ObjNum 126 snap 20 seq_num 126) 2024-09-15T01:33:29.119 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 19 current snap is 29 2024-09-15T01:33:29.119 INFO:tasks.rados.rados.0.smithi175.stdout:333: seq_num 142 ranges {1851392=442368,2293760=720896,3014656=540672,3555328=688128} 2024-09-15T01:33:29.129 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:29.127+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:29.134 INFO:tasks.rados.rados.0.smithi175.stdout:333: writing smithi17529310-19 from 1851392 to 2293760 tid 1 2024-09-15T01:33:29.139 INFO:tasks.rados.rados.0.smithi175.stdout:333: writing smithi17529310-19 from 2293760 to 3014656 tid 2 2024-09-15T01:33:29.143 INFO:tasks.rados.rados.0.smithi175.stdout:333: writing smithi17529310-19 from 3014656 to 3555328 tid 3 2024-09-15T01:33:29.148 INFO:tasks.rados.rados.0.smithi175.stdout:333: writing smithi17529310-19 from 3555328 to 4243456 tid 4 2024-09-15T01:33:29.148 INFO:tasks.rados.rados.0.smithi175.stdout:334: read oid 34 snap 26 2024-09-15T01:33:29.148 INFO:tasks.rados.rados.0.smithi175.stdout:334: expect deleted 2024-09-15T01:33:29.149 INFO:tasks.rados.rados.0.smithi175.stdout:335: snap_create 2024-09-15T01:33:29.149 INFO:tasks.rados.rados.0.smithi175.stdout:330: finishing rollback tid 0 to smithi17529310-45 2024-09-15T01:33:29.149 INFO:tasks.rados.rados.0.smithi175.stdout:330: finishing rollback tid 1 to smithi17529310-45 2024-09-15T01:33:29.149 INFO:tasks.rados.rados.0.smithi175.stdout:330: finishing rollback tid 2 to smithi17529310-45 2024-09-15T01:33:29.149 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 45 v 86 (ObjNum 68 snap 3 seq_num 68) dirty exists 2024-09-15T01:33:29.149 INFO:tasks.rados.rados.0.smithi175.stdout:325: finishing write tid 3 to smithi17529310-39 2024-09-15T01:33:29.149 INFO:tasks.rados.rados.0.smithi175.stdout:325: finishing write tid 4 to smithi17529310-39 2024-09-15T01:33:29.149 INFO:tasks.rados.rados.0.smithi175.stdout:325: finishing write tid 5 to smithi17529310-39 2024-09-15T01:33:29.149 INFO:tasks.rados.rados.0.smithi175.stdout:327: expect (ObjNum 131 snap 21 seq_num 131) 2024-09-15T01:33:29.227 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:29.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:29.227 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:33:29.227 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:33:29.227 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:29.229 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:29.227+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:29.234 INFO:tasks.rados.rados.0.smithi175.stdout:325: finishing write tid 6 to smithi17529310-39 2024-09-15T01:33:29.234 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 39 v 87 (ObjNum 139 snap 29 seq_num 139) dirty exists 2024-09-15T01:33:29.234 INFO:tasks.rados.rados.0.smithi175.stdout:325: left oid 39 (ObjNum 139 snap 29 seq_num 139) 2024-09-15T01:33:29.234 INFO:tasks.rados.rados.0.smithi175.stdout:332: expect (ObjNum 126 snap 20 seq_num 126) 2024-09-15T01:33:29.237 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:33:29.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 reset_purged_snaps_last 2024-09-15T01:33:29.341 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:29.341 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:33:29.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:33:29.341 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:33:29.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.7s3 34/31 ECSubWrite(tid=17, reqid=client.4477.0:77, at_version=34'6, trim_to=0'0, roll_forward_to=34'4))", 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.122771315999998, 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015250150000000001, 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:33:29.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215504+0000", 2024-09-15T01:33:29.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:33:29.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215518+0000", 2024-09-15T01:33:29.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3567000000000001e-05 2024-09-15T01:33:29.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230278+0000", 2024-09-15T01:33:29.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014760300000000001 2024-09-15T01:33:29.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230285+0000", 2024-09-15T01:33:29.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.285e-06 2024-09-15T01:33:29.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230359+0000", 2024-09-15T01:33:29.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4025999999999999e-05 2024-09-15T01:33:29.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230367+0000", 2024-09-15T01:33:29.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4500000000000004e-06 2024-09-15T01:33:29.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230457+0000", 2024-09-15T01:33:29.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0457999999999993e-05 2024-09-15T01:33:29.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230498+0000", 2024-09-15T01:33:29.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0204000000000003e-05 2024-09-15T01:33:29.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230554+0000", 2024-09-15T01:33:29.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6097999999999997e-05 2024-09-15T01:33:29.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230573+0000", 2024-09-15T01:33:29.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9252000000000001e-05 2024-09-15T01:33:29.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.230754+0000", 2024-09-15T01:33:29.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018093300000000001 2024-09-15T01:33:29.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.348 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.348 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:90 3.6s0 3:65d6c4b7:::smithi17529310-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:29.348 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:33:29.348 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.112133397999997, 2024-09-15T01:33:29.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.964412163, 2024-09-15T01:33:29.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:29.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.349 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.349 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 90 2024-09-15T01:33:29.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.349 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:33:29.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226142+0000", 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226144+0000", 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0490000000000002e-06 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226152+0000", 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5700000000000004e-06 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.226157+0000", 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0069999999999997e-06 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.227618+0000", 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014613 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.227698+0000", 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9518000000000007e-05 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.228785+0000", 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010872729999999999 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.228833+0000", 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.85e-05 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:37.179967+0000", 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.95113379099999995 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:37.179971+0000", 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7620000000000001e-06 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:38.180851+0000", 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0008801570000001 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:38.180855+0000", 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.416e-06 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.187397+0000", 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.006542346 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.187673+0000", 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000275825 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:39.190554+0000", 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028811650000000002 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpRead(3.7s3 34/31 ECSubRead(tid=22, to_read={3:e6e8875f:::smithi17529310-31:head=0,4096,0}, subchunks={3:e6e8875f:::smithi17529310-31:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.104761353000001, 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.074844447999999994, 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.355 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233514+0000", 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4300000000000003e-07 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233521+0000", 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0280000000000003e-06 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308171+0000", 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.074650066000000001 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308178+0000", 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5810000000000002e-06 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308258+0000", 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9977999999999999e-05 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308358+0000", 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100152 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.358 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:196 3.6s0 3:65d6c4b7:::smithi17529310-30:head [append 0~0] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 49.135112915999997, 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007181882, 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 196 2024-09-15T01:33:29.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.360 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:33:29.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203162+0000", 2024-09-15T01:33:29.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203168+0000", 2024-09-15T01:33:29.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5620000000000003e-06 2024-09-15T01:33:29.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203175+0000", 2024-09-15T01:33:29.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5599999999999996e-06 2024-09-15T01:33:29.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203194+0000", 2024-09-15T01:33:29.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8757999999999999e-05 2024-09-15T01:33:29.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203201+0000", 2024-09-15T01:33:29.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7429999999999996e-06 2024-09-15T01:33:29.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203414+0000", 2024-09-15T01:33:29.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021300600000000001 2024-09-15T01:33:29.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:29.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203419+0000", 2024-09-15T01:33:29.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1420000000000001e-06 2024-09-15T01:33:29.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.204700+0000", 2024-09-15T01:33:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.001280647 2024-09-15T01:33:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.204945+0000", 2024-09-15T01:33:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024550199999999998 2024-09-15T01:33:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.205894+0000", 2024-09-15T01:33:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094873799999999997 2024-09-15T01:33:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206188+0000", 2024-09-15T01:33:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029375700000000001 2024-09-15T01:33:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206257+0000", 2024-09-15T01:33:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9632e-05 2024-09-15T01:33:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206371+0000", 2024-09-15T01:33:29.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011411100000000001 2024-09-15T01:33:29.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206444+0000", 2024-09-15T01:33:29.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2355999999999996e-05 2024-09-15T01:33:29.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:29.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210281+0000", 2024-09-15T01:33:29.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038376740000000001 2024-09-15T01:33:29.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210344+0000", 2024-09-15T01:33:29.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2694000000000007e-05 2024-09-15T01:33:29.367 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.367 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.367 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.368 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:197 3.6s0 3:65d6c4b7:::smithi17529310-30:head [rollback 3] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:29.368 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:33:29.368 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 49.135059321999996, 2024-09-15T01:33:29.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0090962939999999, 2024-09-15T01:33:29.368 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.368 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:29.368 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.368 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.368 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.368 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 197 2024-09-15T01:33:29.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.369 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:33:29.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:33:29.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203216+0000", 2024-09-15T01:33:29.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8299999999999997e-07 2024-09-15T01:33:29.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203220+0000", 2024-09-15T01:33:29.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0620000000000002e-06 2024-09-15T01:33:29.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203229+0000", 2024-09-15T01:33:29.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5329999999999994e-06 2024-09-15T01:33:29.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203234+0000", 2024-09-15T01:33:29.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000002e-06 2024-09-15T01:33:29.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203537+0000", 2024-09-15T01:33:29.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030286700000000001 2024-09-15T01:33:29.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:29.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203539+0000", 2024-09-15T01:33:29.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.277e-06 2024-09-15T01:33:29.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206748+0000", 2024-09-15T01:33:29.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.003209102 2024-09-15T01:33:29.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.206862+0000", 2024-09-15T01:33:29.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011395000000000001 2024-09-15T01:33:29.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207832+0000", 2024-09-15T01:33:29.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097048299999999996 2024-09-15T01:33:29.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207937+0000", 2024-09-15T01:33:29.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104583 2024-09-15T01:33:29.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.207956+0000", 2024-09-15T01:33:29.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9381e-05 2024-09-15T01:33:29.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208001+0000", 2024-09-15T01:33:29.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4642999999999999e-05 2024-09-15T01:33:29.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209901+0000", 2024-09-15T01:33:29.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001899766 2024-09-15T01:33:29.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:29.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211474+0000", 2024-09-15T01:33:29.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015733190000000001 2024-09-15T01:33:29.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212312+0000", 2024-09-15T01:33:29.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083790500000000001 2024-09-15T01:33:29.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.378 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.378 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:198 3.6s0 3:65d6c4b7:::smithi17529310-30:head [append 0~0] snapc 4=[4,3,2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T01:33:29.378 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:33:29.378 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 49.134775130000001, 2024-09-15T01:33:29.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.009021902, 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 198 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203500+0000", 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203501+0000", 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4600000000000004e-07 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203505+0000", 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3830000000000002e-06 2024-09-15T01:33:29.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203562+0000", 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7120000000000002e-05 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203570+0000", 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203656+0000", 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6273000000000006e-05 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:40.203658+0000", 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4360000000000001e-06 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208296+0000", 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004637574 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.208407+0000", 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110994 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209048+0000", 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064081900000000005 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209141+0000", 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3827999999999997e-05 2024-09-15T01:33:29.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209164+0000", 2024-09-15T01:33:29.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2201e-05 2024-09-15T01:33:29.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209206+0000", 2024-09-15T01:33:29.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2021999999999998e-05 2024-09-15T01:33:29.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.209910+0000", 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070434399999999998 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.211939+0000", 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020290159999999998 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.212522+0000", 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058283599999999999 2024-09-15T01:33:29.385 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:211 3.6s0 3:6ba0b44a:::smithi17529310-6:head [copy-from ver 0] snapc 4=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 48.140294603000001, 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.22039597699999999, 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.386 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197981+0000", 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197982+0000", 2024-09-15T01:33:29.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.618e-06 2024-09-15T01:33:29.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.197988+0000", 2024-09-15T01:33:29.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4519999999999996e-06 2024-09-15T01:33:29.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198027+0000", 2024-09-15T01:33:29.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9492000000000001e-05 2024-09-15T01:33:29.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198036+0000", 2024-09-15T01:33:29.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3950000000000007e-06 2024-09-15T01:33:29.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198736+0000", 2024-09-15T01:33:29.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00069949200000000002 2024-09-15T01:33:29.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:29.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198740+0000", 2024-09-15T01:33:29.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3470000000000001e-06 2024-09-15T01:33:29.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210425+0000", 2024-09-15T01:33:29.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011684905000000001 2024-09-15T01:33:29.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210622+0000", 2024-09-15T01:33:29.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019688099999999999 2024-09-15T01:33:29.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418376+0000", 2024-09-15T01:33:29.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.20775439500000001 2024-09-15T01:33:29.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.392 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.392 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:212 3.6s0 3:6ba0b44a:::smithi17529310-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:33:29.393 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:33:29.393 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 48.140230954000003, 2024-09-15T01:33:29.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.22052332099999999, 2024-09-15T01:33:29.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:29.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.394 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 212 2024-09-15T01:33:29.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.394 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:33:29.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198044+0000", 2024-09-15T01:33:29.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198045+0000", 2024-09-15T01:33:29.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6499999999999998e-07 2024-09-15T01:33:29.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198049+0000", 2024-09-15T01:33:29.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4150000000000001e-06 2024-09-15T01:33:29.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198058+0000", 2024-09-15T01:33:29.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9830000000000002e-06 2024-09-15T01:33:29.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198064+0000", 2024-09-15T01:33:29.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6219999999999997e-06 2024-09-15T01:33:29.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198782+0000", 2024-09-15T01:33:29.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071748199999999997 2024-09-15T01:33:29.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:29.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.198783+0000", 2024-09-15T01:33:29.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9400000000000001e-06 2024-09-15T01:33:29.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210808+0000", 2024-09-15T01:33:29.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012024217 2024-09-15T01:33:29.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:29.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.210861+0000", 2024-09-15T01:33:29.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2859999999999999e-05 2024-09-15T01:33:29.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418443+0000", 2024-09-15T01:33:29.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.20758253099999999 2024-09-15T01:33:29.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418509+0000", 2024-09-15T01:33:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5516000000000001e-05 2024-09-15T01:33:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.418567+0000", 2024-09-15T01:33:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.889e-05 2024-09-15T01:33:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs6 39/31 ECSubWrite(tid=56, reqid=osd.11.0:56, at_version=39'21, trim_to=0'0, roll_forward_to=38'17))", 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 48.095482201999999, 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038235071000000002, 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242793+0000", 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242794+0000", 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8599999999999997e-07 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.242817+0000", 2024-09-15T01:33:29.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3655e-05 2024-09-15T01:33:29.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280383+0000", 2024-09-15T01:33:29.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037566100999999998 2024-09-15T01:33:29.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280390+0000", 2024-09-15T01:33:29.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:33:29.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280442+0000", 2024-09-15T01:33:29.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1650000000000002e-05 2024-09-15T01:33:29.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280448+0000", 2024-09-15T01:33:29.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7479999999999999e-06 2024-09-15T01:33:29.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280660+0000", 2024-09-15T01:33:29.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021205900000000001 2024-09-15T01:33:29.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280667+0000", 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4710000000000001e-06 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280705+0000", 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7511000000000001e-05 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.280722+0000", 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7646999999999999e-05 2024-09-15T01:33:29.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:41.281028+0000", 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030575099999999999 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs2 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.095960130000002, 2024-09-15T01:33:29.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011429094000000001, 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242315+0000", 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242316+0000", 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2900000000000003e-07 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242337+0000", 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1263999999999998e-05 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242392+0000", 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5408999999999998e-05 2024-09-15T01:33:29.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242399+0000", 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.866e-06 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242456+0000", 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-05 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242466+0000", 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0015e-05 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242685+0000", 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021821399999999999 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242713+0000", 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8589e-05 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242754+0000", 2024-09-15T01:33:29.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1173e-05 2024-09-15T01:33:29.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242790+0000", 2024-09-15T01:33:29.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5167999999999999e-05 2024-09-15T01:33:29.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253744+0000", 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010954527 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs1 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.095463746, 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010913366000000001, 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.414 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242811+0000", 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242812+0000", 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242834+0000", 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1827999999999998e-05 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242901+0000", 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6840000000000004e-05 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242908+0000", 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5440000000000001e-06 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242959+0000", 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1057000000000003e-05 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.242969+0000", 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8030000000000008e-06 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243189+0000", 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021987499999999999 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243212+0000", 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2569999999999999e-05 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243258+0000", 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6452000000000002e-05 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243274+0000", 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6426000000000002e-05 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.253725+0000", 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010450249 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs6 44/31 ECSubWrite(tid=82, reqid=client.4477.0:281, at_version=44'36, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:33:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.094710663999997, 2024-09-15T01:33:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011937237, 2024-09-15T01:33:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:33:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243564+0000", 2024-09-15T01:33:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243565+0000", 2024-09-15T01:33:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0500000000000003e-07 2024-09-15T01:33:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243585+0000", 2024-09-15T01:33:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0313e-05 2024-09-15T01:33:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243689+0000", 2024-09-15T01:33:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103977 2024-09-15T01:33:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243696+0000", 2024-09-15T01:33:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0489999999999998e-06 2024-09-15T01:33:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243791+0000", 2024-09-15T01:33:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4394000000000005e-05 2024-09-15T01:33:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.243800+0000", 2024-09-15T01:33:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2439999999999999e-06 2024-09-15T01:33:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244285+0000", 2024-09-15T01:33:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000484802 2024-09-15T01:33:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244313+0000", 2024-09-15T01:33:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8515000000000001e-05 2024-09-15T01:33:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244359+0000", 2024-09-15T01:33:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5389000000000002e-05 2024-09-15T01:33:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.244375+0000", 2024-09-15T01:33:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6699000000000001e-05 2024-09-15T01:33:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.255502+0000", 2024-09-15T01:33:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011126250000000001 2024-09-15T01:33:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 44/31 ECSubWrite(tid=39, reqid=client.4477.0:283, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:33:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.069988674000001, 2024-09-15T01:33:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012672954, 2024-09-15T01:33:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:33:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268286+0000", 2024-09-15T01:33:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268287+0000", 2024-09-15T01:33:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.85e-07 2024-09-15T01:33:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.268806+0000", 2024-09-15T01:33:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00051859699999999998 2024-09-15T01:33:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272018+0000", 2024-09-15T01:33:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032124620000000001 2024-09-15T01:33:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272032+0000", 2024-09-15T01:33:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4008e-05 2024-09-15T01:33:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272076+0000", 2024-09-15T01:33:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3807999999999998e-05 2024-09-15T01:33:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272085+0000", 2024-09-15T01:33:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9749999999999996e-06 2024-09-15T01:33:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272331+0000", 2024-09-15T01:33:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024564600000000003 2024-09-15T01:33:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272349+0000", 2024-09-15T01:33:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7988000000000001e-05 2024-09-15T01:33:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272412+0000", 2024-09-15T01:33:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2718000000000005e-05 2024-09-15T01:33:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.272426+0000", 2024-09-15T01:33:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:33:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:46.280959+0000", 2024-09-15T01:33:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085329199999999994 2024-09-15T01:33:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s1 55/31 ECSubWrite(tid=67, reqid=client.4477.0:503, at_version=0'0, trim_to=0'0, roll_forward_to=55'51))", 2024-09-15T01:33:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:57.106011+0000", 2024-09-15T01:33:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.232264262000001, 2024-09-15T01:33:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017261852000000001, 2024-09-15T01:33:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.106011+0000", 2024-09-15T01:33:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.106011+0000", 2024-09-15T01:33:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.106013+0000", 2024-09-15T01:33:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2079999999999999e-06 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.106037+0000", 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4114e-05 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122822+0000", 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016784701999999999 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122834+0000", 2024-09-15T01:33:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2055000000000001e-05 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122891+0000", 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7244999999999998e-05 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122899+0000", 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.122994+0000", 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4554000000000003e-05 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.123004+0000", 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.049e-05 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.123055+0000", 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0927999999999998e-05 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.123074+0000", 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9177999999999998e-05 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.123273+0000", 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019858899999999999 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 60/38 ECSubWriteReply(tid=51, last_complete=60'31, committed=1, applied=1))", 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:02.100140+0000", 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.238134873, 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.402923223, 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.100140+0000", 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.100140+0000", 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.100141+0000", 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9599999999999999e-07 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.100145+0000", 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1949999999999996e-06 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.502953+0000", 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.40280827200000002 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.502961+0000", 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9840000000000001e-06 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.503036+0000", 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4394999999999995e-05 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:02.503063+0000", 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7781e-05 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:636 3.6s0 3:6ba0b44a:::smithi17529310-6:head [copy-from ver 36] snapc 21=[21,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:11.095326+0000", 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.242948706, 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0117660310000001, 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 636 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095326+0000", 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095326+0000", 2024-09-15T01:33:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095328+0000", 2024-09-15T01:33:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3859999999999999e-06 2024-09-15T01:33:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095333+0000", 2024-09-15T01:33:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0660000000000003e-06 2024-09-15T01:33:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095345+0000", 2024-09-15T01:33:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1979e-05 2024-09-15T01:33:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095350+0000", 2024-09-15T01:33:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7400000000000004e-06 2024-09-15T01:33:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095405+0000", 2024-09-15T01:33:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5606999999999998e-05 2024-09-15T01:33:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095408+0000", 2024-09-15T01:33:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7520000000000001e-06 2024-09-15T01:33:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.081910+0000", 2024-09-15T01:33:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.98650184200000002 2024-09-15T01:33:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.082126+0000", 2024-09-15T01:33:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021671099999999999 2024-09-15T01:33:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102015+0000", 2024-09-15T01:33:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019888449999999998 2024-09-15T01:33:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102256+0000", 2024-09-15T01:33:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024061300000000001 2024-09-15T01:33:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102304+0000", 2024-09-15T01:33:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8597000000000002e-05 2024-09-15T01:33:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102756+0000", 2024-09-15T01:33:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045214000000000002 2024-09-15T01:33:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102920+0000", 2024-09-15T01:33:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000163793 2024-09-15T01:33:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.107042+0000", 2024-09-15T01:33:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041217019999999997 2024-09-15T01:33:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.107092+0000", 2024-09-15T01:33:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0652999999999998e-05 2024-09-15T01:33:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:637 3.6s0 3:6ba0b44a:::smithi17529310-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-15T01:33:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:11.095356+0000", 2024-09-15T01:33:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.242918614000001, 2024-09-15T01:33:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0073326149999999, 2024-09-15T01:33:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 637 2024-09-15T01:33:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095356+0000", 2024-09-15T01:33:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095356+0000", 2024-09-15T01:33:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095357+0000", 2024-09-15T01:33:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-15T01:33:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095358+0000", 2024-09-15T01:33:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4130000000000001e-06 2024-09-15T01:33:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095362+0000", 2024-09-15T01:33:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0459999999999999e-06 2024-09-15T01:33:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095365+0000", 2024-09-15T01:33:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5409999999999999e-06 2024-09-15T01:33:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095430+0000", 2024-09-15T01:33:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5026999999999999e-05 2024-09-15T01:33:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-15T01:33:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:11.095431+0000", 2024-09-15T01:33:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0899999999999999e-06 2024-09-15T01:33:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.084105+0000", 2024-09-15T01:33:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.98867371999999998 2024-09-15T01:33:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.084176+0000", 2024-09-15T01:33:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0876000000000004e-05 2024-09-15T01:33:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102436+0000", 2024-09-15T01:33:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018259958 2024-09-15T01:33:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102560+0000", 2024-09-15T01:33:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012495599999999999 2024-09-15T01:33:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:12.102689+0000", 2024-09-15T01:33:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012861900000000001 2024-09-15T01:33:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:701 3.6s0 3:6ba0b44a:::smithi17529310-6:head [copy-from ver 59] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.298859+0000", 2024-09-15T01:33:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.039416507999999, 2024-09-15T01:33:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.031040582000000001, 2024-09-15T01:33:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 701 2024-09-15T01:33:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298859+0000", 2024-09-15T01:33:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298859+0000", 2024-09-15T01:33:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298860+0000", 2024-09-15T01:33:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8530000000000001e-06 2024-09-15T01:33:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298868+0000", 2024-09-15T01:33:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1969999999999999e-06 2024-09-15T01:33:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298884+0000", 2024-09-15T01:33:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6688999999999999e-05 2024-09-15T01:33:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298891+0000", 2024-09-15T01:33:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6290000000000004e-06 2024-09-15T01:33:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298945+0000", 2024-09-15T01:33:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3533999999999999e-05 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.299722+0000", 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077747099999999996 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321205+0000", 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021483149999999999 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321806+0000", 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060049599999999995 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.321829+0000", 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3229000000000001e-05 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322419+0000", 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058996099999999996 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322847+0000", 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042865900000000002 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.329826+0000", 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069782359999999996 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.329899+0000", 2024-09-15T01:33:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3478000000000004e-05 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:702 3.6s0 3:6ba0b44a:::smithi17529310-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:13.298904+0000", 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.039371419999998, 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023906357, 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 702 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298904+0000", 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298904+0000", 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298904+0000", 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6199999999999998e-07 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298907+0000", 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2179999999999999e-06 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298915+0000", 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.943e-06 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.298919+0000", 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0160000000000002e-06 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.300003+0000", 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010835440000000001 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.300092+0000", 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8721000000000007e-05 2024-09-15T01:33:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322365+0000", 2024-09-15T01:33:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022273173 2024-09-15T01:33:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322606+0000", 2024-09-15T01:33:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024124699999999999 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:13.322810+0000", 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000203933 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs6 76/31 ECSubWrite(tid=279, reqid=client.4477.0:751, at_version=76'153, trim_to=0'0, roll_forward_to=73'147))", 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:18.088335+0000", 2024-09-15T01:33:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.249940435999999, 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01095546, 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088335+0000", 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088335+0000", 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088335+0000", 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4399999999999999e-07 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088355+0000", 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0604000000000001e-05 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088849+0000", 2024-09-15T01:33:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049329899999999999 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088855+0000", 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088918+0000", 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2182999999999997e-05 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.088929+0000", 2024-09-15T01:33:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1270000000000001e-05 2024-09-15T01:33:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.089476+0000", 2024-09-15T01:33:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00054738699999999998 2024-09-15T01:33:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.089514+0000", 2024-09-15T01:33:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7858999999999998e-05 2024-09-15T01:33:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.089563+0000", 2024-09-15T01:33:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8421999999999998e-05 2024-09-15T01:33:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.089621+0000", 2024-09-15T01:33:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8103999999999998e-05 2024-09-15T01:33:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.099290+0000", 2024-09-15T01:33:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0096694829999999996 2024-09-15T01:33:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.479 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:29.480 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:29.339+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:29.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:33:29.482 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 reset_purged_snaps_last 2024-09-15T01:33:29.483 INFO:tasks.rados.rados.0.smithi175.stdout:333: finishing write tid 1 to smithi17529310-19 2024-09-15T01:33:29.483 INFO:tasks.rados.rados.0.smithi175.stdout:333: finishing write tid 2 to smithi17529310-19 2024-09-15T01:33:29.483 INFO:tasks.rados.rados.0.smithi175.stdout:333: finishing write tid 3 to smithi17529310-19 2024-09-15T01:33:29.483 INFO:tasks.rados.rados.0.smithi175.stdout:333: finishing write tid 4 to smithi17529310-19 2024-09-15T01:33:29.483 INFO:tasks.rados.rados.0.smithi175.stdout:333: finishing write tid 5 to smithi17529310-19 2024-09-15T01:33:29.483 INFO:tasks.rados.rados.0.smithi175.stdout:333: finishing write tid 6 to smithi17529310-19 2024-09-15T01:33:29.483 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 19 v 103 (ObjNum 142 snap 29 seq_num 142) dirty exists 2024-09-15T01:33:29.483 INFO:tasks.rados.rados.0.smithi175.stdout:333: left oid 19 (ObjNum 142 snap 29 seq_num 142) 2024-09-15T01:33:29.484 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:29.480+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:29.584 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:29.583+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:29.624 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-15T01:33:29.625 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:29.633 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:33:29.664 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 reset_purged_snaps_last 2024-09-15T01:33:29.706 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:29.704+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:29.736 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:33:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:33:29.736 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:29.745 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:33:29.786 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:29.784+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:29.847 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 reset_purged_snaps_last 2024-09-15T01:33:29.886 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:29.885+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:29.890 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:29.890 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-15T01:33:29.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-15T01:33:29.890 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:33:29.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.890 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:14 3.1s0 3:8930608f:::smithi17529310-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:29.890 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.176805+0000", 2024-09-15T01:33:29.890 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.710437528, 2024-09-15T01:33:29.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020710388, 2024-09-15T01:33:29.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:29.890 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.891 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.891 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 14 2024-09-15T01:33:29.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.891 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176805+0000", 2024-09-15T01:33:29.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176805+0000", 2024-09-15T01:33:29.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176806+0000", 2024-09-15T01:33:29.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-15T01:33:29.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176810+0000", 2024-09-15T01:33:29.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8399999999999997e-06 2024-09-15T01:33:29.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176824+0000", 2024-09-15T01:33:29.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4051e-05 2024-09-15T01:33:29.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176827+0000", 2024-09-15T01:33:29.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5159999999999999e-06 2024-09-15T01:33:29.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178153+0000", 2024-09-15T01:33:29.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013253519999999999 2024-09-15T01:33:29.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178241+0000", 2024-09-15T01:33:29.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8670000000000003e-05 2024-09-15T01:33:29.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178708+0000", 2024-09-15T01:33:29.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00046673299999999999 2024-09-15T01:33:29.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178860+0000", 2024-09-15T01:33:29.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015201099999999999 2024-09-15T01:33:29.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179098+0000", 2024-09-15T01:33:29.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238123 2024-09-15T01:33:29.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179138+0000", 2024-09-15T01:33:29.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0184e-05 2024-09-15T01:33:29.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180215+0000", 2024-09-15T01:33:29.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001076544 2024-09-15T01:33:29.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:29.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197335+0000", 2024-09-15T01:33:29.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017120209000000001 2024-09-15T01:33:29.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197516+0000", 2024-09-15T01:33:29.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000180657 2024-09-15T01:33:29.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.898 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.898 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:15 3.1s0 3:8930608f:::smithi17529310-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:29.898 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.176858+0000", 2024-09-15T01:33:29.898 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.710384441000002, 2024-09-15T01:33:29.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023897231000000001, 2024-09-15T01:33:29.899 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.899 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:29.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15 2024-09-15T01:33:29.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.899 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176858+0000", 2024-09-15T01:33:29.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176858+0000", 2024-09-15T01:33:29.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176859+0000", 2024-09-15T01:33:29.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0999999999999999e-07 2024-09-15T01:33:29.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176861+0000", 2024-09-15T01:33:29.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4099999999999998e-06 2024-09-15T01:33:29.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176875+0000", 2024-09-15T01:33:29.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3366e-05 2024-09-15T01:33:29.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.176882+0000", 2024-09-15T01:33:29.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5560000000000002e-06 2024-09-15T01:33:29.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178931+0000", 2024-09-15T01:33:29.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002048299 2024-09-15T01:33:29.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:29.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179014+0000", 2024-09-15T01:33:29.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3207000000000005e-05 2024-09-15T01:33:29.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197559+0000", 2024-09-15T01:33:29.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018545582000000001 2024-09-15T01:33:29.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197656+0000", 2024-09-15T01:33:29.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7085999999999998e-05 2024-09-15T01:33:29.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.200756+0000", 2024-09-15T01:33:29.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030992150000000002 2024-09-15T01:33:29.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.904 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.905 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:29 3.7s0 3:e2b4e7d9:::smithi17529310-10:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:29.905 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.177051+0000", 2024-09-15T01:33:29.905 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.710191494999997, 2024-09-15T01:33:29.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020495925000000002, 2024-09-15T01:33:29.905 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.905 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:29.905 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.905 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.905 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.905 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 29 2024-09-15T01:33:29.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.906 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177051+0000", 2024-09-15T01:33:29.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177051+0000", 2024-09-15T01:33:29.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177052+0000", 2024-09-15T01:33:29.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-15T01:33:29.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177064+0000", 2024-09-15T01:33:29.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1477999999999999e-05 2024-09-15T01:33:29.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177072+0000", 2024-09-15T01:33:29.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5629999999999992e-06 2024-09-15T01:33:29.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177084+0000", 2024-09-15T01:33:29.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1557e-05 2024-09-15T01:33:29.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178451+0000", 2024-09-15T01:33:29.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013673019999999999 2024-09-15T01:33:29.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178548+0000", 2024-09-15T01:33:29.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6639999999999996e-05 2024-09-15T01:33:29.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179029+0000", 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048111300000000003 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179171+0000", 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014193800000000001 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179183+0000", 2024-09-15T01:33:29.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2196e-05 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179212+0000", 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9462999999999999e-05 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179251+0000", 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9155e-05 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197330+0000", 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018078409 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197547+0000", 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021747699999999999 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.912 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:30 3.7s0 3:e2b4e7d9:::smithi17529310-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:29.913 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.177099+0000", 2024-09-15T01:33:29.913 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.710143756999997, 2024-09-15T01:33:29.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024179603000000001, 2024-09-15T01:33:29.913 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.913 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:29.913 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.913 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.913 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.913 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 30 2024-09-15T01:33:29.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.913 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177099+0000", 2024-09-15T01:33:29.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177099+0000", 2024-09-15T01:33:29.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177100+0000", 2024-09-15T01:33:29.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-07 2024-09-15T01:33:29.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177103+0000", 2024-09-15T01:33:29.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.847e-06 2024-09-15T01:33:29.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177111+0000", 2024-09-15T01:33:29.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4780000000000008e-06 2024-09-15T01:33:29.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177116+0000", 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999998e-06 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179311+0000", 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002195331 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:29.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179405+0000", 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4611999999999995e-05 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197589+0000", 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018183300999999999 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.197665+0000", 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5878999999999996e-05 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.201279+0000", 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003614089 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:31 3.1s0 3:8776790e:::smithi17529310-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.177756+0000", 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.709487121999999, 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020342740000000002, 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:29.919 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.919 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.919 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.919 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 31 2024-09-15T01:33:29.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.919 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177756+0000", 2024-09-15T01:33:29.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177756+0000", 2024-09-15T01:33:29.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.177757+0000", 2024-09-15T01:33:29.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-15T01:33:29.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178059+0000", 2024-09-15T01:33:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030214800000000002 2024-09-15T01:33:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178069+0000", 2024-09-15T01:33:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0324e-05 2024-09-15T01:33:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178081+0000", 2024-09-15T01:33:29.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1491999999999999e-05 2024-09-15T01:33:29.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179041+0000", 2024-09-15T01:33:29.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096040399999999999 2024-09-15T01:33:29.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.179333+0000", 2024-09-15T01:33:29.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029163799999999999 2024-09-15T01:33:29.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180034+0000", 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070128900000000001 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180155+0000", 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120844 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180166+0000", 2024-09-15T01:33:29.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0967e-05 2024-09-15T01:33:29.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180220+0000", 2024-09-15T01:33:29.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4048999999999997e-05 2024-09-15T01:33:29.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182483+0000", 2024-09-15T01:33:29.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022628930000000002 2024-09-15T01:33:29.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.198037+0000", 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015553726 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.198099+0000", 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1824999999999998e-05 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:32 3.1s0 3:8776790e:::smithi17529310-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.178085+0000", 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.709157527000002, 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021154846000000001, 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.926 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178085+0000", 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178085+0000", 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178086+0000", 2024-09-15T01:33:29.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0699999999999997e-07 2024-09-15T01:33:29.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178088+0000", 2024-09-15T01:33:29.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1100000000000001e-06 2024-09-15T01:33:29.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178096+0000", 2024-09-15T01:33:29.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4419999999999998e-06 2024-09-15T01:33:29.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178101+0000", 2024-09-15T01:33:29.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1810000000000003e-06 2024-09-15T01:33:29.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180259+0000", 2024-09-15T01:33:29.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021580839999999998 2024-09-15T01:33:29.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.180359+0000", 2024-09-15T01:33:29.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100028 2024-09-15T01:33:29.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181451+0000", 2024-09-15T01:33:29.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001092569 2024-09-15T01:33:29.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181638+0000", 2024-09-15T01:33:29.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018707899999999999 2024-09-15T01:33:29.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.181720+0000", 2024-09-15T01:33:29.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1682999999999997e-05 2024-09-15T01:33:29.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182311+0000", 2024-09-15T01:33:29.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059133299999999998 2024-09-15T01:33:29.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182487+0000", 2024-09-15T01:33:29.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017578299999999999 2024-09-15T01:33:29.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:29.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199166+0000", 2024-09-15T01:33:29.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016678622000000001 2024-09-15T01:33:29.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199240+0000", 2024-09-15T01:33:29.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4425000000000007e-05 2024-09-15T01:33:29.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.934 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.934 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:33 3.1s0 3:8776790e:::smithi17529310-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:29.934 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:33:29.934 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.709138103000001, 2024-09-15T01:33:29.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024100798, 2024-09-15T01:33:29.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:29.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 33 2024-09-15T01:33:29.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.935 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:33:29.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:33:29.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178105+0000", 2024-09-15T01:33:29.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-15T01:33:29.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178108+0000", 2024-09-15T01:33:29.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3259999999999998e-06 2024-09-15T01:33:29.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178114+0000", 2024-09-15T01:33:29.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5209999999999999e-06 2024-09-15T01:33:29.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.178118+0000", 2024-09-15T01:33:29.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7139999999999999e-06 2024-09-15T01:33:29.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182355+0000", 2024-09-15T01:33:29.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042371070000000004 2024-09-15T01:33:29.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:29.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.182467+0000", 2024-09-15T01:33:29.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011186799999999999 2024-09-15T01:33:29.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199900+0000", 2024-09-15T01:33:29.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017433015999999999 2024-09-15T01:33:29.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.199973+0000", 2024-09-15T01:33:29.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2897000000000004e-05 2024-09-15T01:33:29.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.202206+0000", 2024-09-15T01:33:29.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022329049999999999 2024-09-15T01:33:29.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.940 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:44 3.bs0 3:d6f740a8:::smithi17529310-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.185918+0000", 2024-09-15T01:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.701325056999998, 2024-09-15T01:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.123698503, 2024-09-15T01:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44 2024-09-15T01:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185918+0000", 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185918+0000", 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185919+0000", 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7400000000000003e-07 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185927+0000", 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7860000000000002e-06 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185936+0000", 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7569999999999993e-06 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185941+0000", 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1889999999999998e-06 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187237+0000", 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001295806 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187337+0000", 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9630000000000007e-05 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187837+0000", 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000500757 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187919+0000", 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1482000000000004e-05 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187962+0000", 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2647e-05 2024-09-15T01:33:29.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188001+0000", 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9070999999999999e-05 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188078+0000", 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7713999999999995e-05 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:29.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309538+0000", 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.121459186 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309616+0000", 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8804e-05 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:45 3.bs0 3:d6f740a8:::smithi17529310-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:33:29.947 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.701297283000002, 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12623172299999999, 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 45 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185946+0000", 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185948+0000", 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-15T01:33:29.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185954+0000", 2024-09-15T01:33:29.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1090000000000003e-06 2024-09-15T01:33:29.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.185958+0000", 2024-09-15T01:33:29.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5159999999999999e-06 2024-09-15T01:33:29.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.187975+0000", 2024-09-15T01:33:29.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002017343 2024-09-15T01:33:29.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:29.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.188054+0000", 2024-09-15T01:33:29.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8734999999999997e-05 2024-09-15T01:33:29.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309670+0000", 2024-09-15T01:33:29.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.121615926 2024-09-15T01:33:29.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.309756+0000", 2024-09-15T01:33:29.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6316999999999994e-05 2024-09-15T01:33:29.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.312177+0000", 2024-09-15T01:33:29.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002421126 2024-09-15T01:33:29.952 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:78 3.7s0 3:eb933b0c:::smithi17529310-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.213590+0000", 2024-09-15T01:33:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.673652709000002, 2024-09-15T01:33:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020370672999999999, 2024-09-15T01:33:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.954 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.954 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.954 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 78 2024-09-15T01:33:29.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.954 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213590+0000", 2024-09-15T01:33:29.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213590+0000", 2024-09-15T01:33:29.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213591+0000", 2024-09-15T01:33:29.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:33:29.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213593+0000", 2024-09-15T01:33:29.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3199999999999998e-06 2024-09-15T01:33:29.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213600+0000", 2024-09-15T01:33:29.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4580000000000004e-06 2024-09-15T01:33:29.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.213607+0000", 2024-09-15T01:33:29.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-06 2024-09-15T01:33:29.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215631+0000", 2024-09-15T01:33:29.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020248890000000002 2024-09-15T01:33:29.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:29.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.215718+0000", 2024-09-15T01:33:29.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6127999999999999e-05 2024-09-15T01:33:29.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231591+0000", 2024-09-15T01:33:29.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015873015000000001 2024-09-15T01:33:29.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231666+0000", 2024-09-15T01:33:29.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5525999999999995e-05 2024-09-15T01:33:29.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233961+0000", 2024-09-15T01:33:29.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022948069999999998 2024-09-15T01:33:29.959 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.959 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.959 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.959 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:93 3.7s0 3:e6e8875f:::smithi17529310-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:29.959 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:33:29.959 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.657857374999999, 2024-09-15T01:33:29.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079343245000000007, 2024-09-15T01:33:29.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.960 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:29.960 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.960 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.960 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.960 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 93 2024-09-15T01:33:29.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.960 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:33:29.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:33:29.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229386+0000", 2024-09-15T01:33:29.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1299999999999999e-07 2024-09-15T01:33:29.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229390+0000", 2024-09-15T01:33:29.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.703e-06 2024-09-15T01:33:29.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229403+0000", 2024-09-15T01:33:29.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3056e-05 2024-09-15T01:33:29.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.229410+0000", 2024-09-15T01:33:29.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-15T01:33:29.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231135+0000", 2024-09-15T01:33:29.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001725737 2024-09-15T01:33:29.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.231216+0000", 2024-09-15T01:33:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0586000000000004e-05 2024-09-15T01:33:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233251+0000", 2024-09-15T01:33:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020346819999999999 2024-09-15T01:33:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.233339+0000", 2024-09-15T01:33:29.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7907999999999999e-05 2024-09-15T01:33:29.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:36.308729+0000", 2024-09-15T01:33:29.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.075390190999999995 2024-09-15T01:33:29.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.965 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.7s0 47/38 ECSubWriteReply(tid=76, last_complete=47'25, committed=1, applied=1))", 2024-09-15T01:33:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:33:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.523020365999997, 2024-09-15T01:33:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.42564722900000002, 2024-09-15T01:33:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:33:29.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:33:29.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364223+0000", 2024-09-15T01:33:29.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8599999999999999e-07 2024-09-15T01:33:29.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.364224+0000", 2024-09-15T01:33:29.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5099999999999999e-06 2024-09-15T01:33:29.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789796+0000", 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.425571957 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789805+0000", 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0270000000000001e-06 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789857+0000", 2024-09-15T01:33:29.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1598999999999998e-05 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:49.789870+0000", 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2850000000000001e-05 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpReadReply(3.1s0 53/52 ECSubReadReply(tid=94, attrs_read=0))", 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:55.205063+0000", 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.682179435999998, 2024-09-15T01:33:29.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99844881399999996, 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:55.205063+0000", 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:55.205063+0000", 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:55.205064+0000", 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:55.205072+0000", 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5299999999999996e-06 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.202784+0000", 2024-09-15T01:33:29.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99771169100000001 2024-09-15T01:33:29.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.202787+0000", 2024-09-15T01:33:29.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.841e-06 2024-09-15T01:33:29.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.202869+0000", 2024-09-15T01:33:29.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1705999999999993e-05 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.203512+0000", 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064375100000000002 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpReadReply(3.7s0 54/52 ECSubReadReply(tid=102, attrs_read=0))", 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.107553+0000", 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.779689681000001, 2024-09-15T01:33:29.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.96435113900000002, 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.107553+0000", 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.107553+0000", 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.107554+0000", 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3500000000000002e-07 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.107690+0000", 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000136157 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.071644+0000", 2024-09-15T01:33:29.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.96395441400000004 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.071651+0000", 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0419999999999997e-06 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.071704+0000", 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2580000000000001e-05 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.071904+0000", 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020051100000000001 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpReadReply(3.1s0 54/52 ECSubReadReply(tid=116, attrs_read=0))", 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.207373+0000", 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.679869441000001, 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.83479060699999996, 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.207373+0000", 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.207373+0000", 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.207374+0000", 2024-09-15T01:33:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8599999999999998e-07 2024-09-15T01:33:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.207523+0000", 2024-09-15T01:33:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014929200000000001 2024-09-15T01:33:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.041829+0000", 2024-09-15T01:33:29.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.83430607300000004 2024-09-15T01:33:29.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.041837+0000", 2024-09-15T01:33:29.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9030000000000004e-06 2024-09-15T01:33:29.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.041884+0000", 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6993999999999998e-05 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:57.042164+0000", 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027975899999999998 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:500 3.7s0 3:e89d09e8:::smithi17529310-19:head [setxattr _header (57) in=64b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:29.982 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.209208+0000", 2024-09-15T01:33:29.983 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.678034697000001, 2024-09-15T01:33:29.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.280345175, 2024-09-15T01:33:29.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:29.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 500 2024-09-15T01:33:29.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.983 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209208+0000", 2024-09-15T01:33:29.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209208+0000", 2024-09-15T01:33:29.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209209+0000", 2024-09-15T01:33:29.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4199999999999995e-07 2024-09-15T01:33:29.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209213+0000", 2024-09-15T01:33:29.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8410000000000002e-06 2024-09-15T01:33:29.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209217+0000", 2024-09-15T01:33:29.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.391e-06 2024-09-15T01:33:29.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209220+0000", 2024-09-15T01:33:29.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6570000000000001e-06 2024-09-15T01:33:29.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.210612+0000", 2024-09-15T01:33:29.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013926850000000001 2024-09-15T01:33:29.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.210830+0000", 2024-09-15T01:33:29.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000217655 2024-09-15T01:33:29.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:29.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211263+0000", 2024-09-15T01:33:29.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043276600000000001 2024-09-15T01:33:29.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:29.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211409+0000", 2024-09-15T01:33:29.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014583400000000001 2024-09-15T01:33:29.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:29.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211418+0000", 2024-09-15T01:33:29.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7459999999999995e-06 2024-09-15T01:33:29.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:29.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211458+0000", 2024-09-15T01:33:29.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999999e-05 2024-09-15T01:33:29.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:29.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211653+0000", 2024-09-15T01:33:29.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000195801 2024-09-15T01:33:29.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:29.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.489314+0000", 2024-09-15T01:33:29.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.27766080300000001 2024-09-15T01:33:29.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.489553+0000", 2024-09-15T01:33:29.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023910399999999999 2024-09-15T01:33:29.990 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.990 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.991 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:501 3.7s0 3:e89d09e8:::smithi17529310-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:29.991 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:33:29.991 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.678020340000003, 2024-09-15T01:33:29.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.28201526999999998, 2024-09-15T01:33:29.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:29.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:29.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:29.992 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:29.992 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 501 2024-09-15T01:33:29.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.992 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:33:29.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:33:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209223+0000", 2024-09-15T01:33:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-15T01:33:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209225+0000", 2024-09-15T01:33:29.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9819999999999998e-06 2024-09-15T01:33:29.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:29.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209229+0000", 2024-09-15T01:33:29.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3959999999999999e-06 2024-09-15T01:33:29.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:29.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.209233+0000", 2024-09-15T01:33:29.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5609999999999999e-06 2024-09-15T01:33:29.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211492+0000", 2024-09-15T01:33:29.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002259414 2024-09-15T01:33:29.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:29.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211637+0000", 2024-09-15T01:33:29.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014431099999999999 2024-09-15T01:33:29.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:29.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.489907+0000", 2024-09-15T01:33:29.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.27827002099999998 2024-09-15T01:33:29.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:29.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.490070+0000", 2024-09-15T01:33:29.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016353799999999999 2024-09-15T01:33:29.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:29.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.491238+0000", 2024-09-15T01:33:29.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011676309999999999 2024-09-15T01:33:29.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.997 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:29.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:29.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.997 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.7s0 54/52 ECSubWriteReply(tid=119, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:29.997 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:32:56.211852+0000", 2024-09-15T01:33:29.998 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.675390978999999, 2024-09-15T01:33:29.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19905882499999999, 2024-09-15T01:33:29.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:29.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-15T01:33:29.998 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:29.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:29.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211852+0000", 2024-09-15T01:33:29.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211852+0000", 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211853+0000", 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8500000000000001e-07 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.211858+0000", 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9049999999999996e-06 2024-09-15T01:33:29.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.410806+0000", 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.198948825 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.410818+0000", 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1161e-05 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.410890+0000", 2024-09-15T01:33:30.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2223000000000003e-05 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:32:56.410911+0000", 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1026000000000002e-05 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:748 3.7s0 3:e89d09e8:::smithi17529310-19:head [copy-from ver 67] snapc 27=[27,26,24,22,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:18.077994+0000", 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.809248773, 2024-09-15T01:33:30.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.031699640000000001, 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 748 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.077994+0000", 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.077994+0000", 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.077996+0000", 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.339e-06 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078006+0000", 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9359999999999994e-06 2024-09-15T01:33:30.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078024+0000", 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7912999999999999e-05 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078032+0000", 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9549999999999992e-06 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:30.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078096+0000", 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3443999999999998e-05 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078104+0000", 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0169999999999996e-06 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.079874+0000", 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001769851 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.079984+0000", 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110844 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.103104+0000", 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023119681 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.103642+0000", 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000537906 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:30.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.103661+0000", 2024-09-15T01:33:30.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8632999999999999e-05 2024-09-15T01:33:30.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:30.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.103968+0000", 2024-09-15T01:33:30.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030692300000000002 2024-09-15T01:33:30.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-09-15T01:33:30.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.104214+0000", 2024-09-15T01:33:30.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024635400000000001 2024-09-15T01:33:30.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-15T01:33:30.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.106612+0000", 2024-09-15T01:33:30.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00239756 2024-09-15T01:33:30.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:30.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.109694+0000", 2024-09-15T01:33:30.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030822839999999998 2024-09-15T01:33:30.009 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:30.009 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:30.009 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:30.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.009 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:749 3.7s0 3:e89d09e8:::smithi17529310-19:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e76)", 2024-09-15T01:33:30.009 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:18.078047+0000", 2024-09-15T01:33:30.009 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.809195896, 2024-09-15T01:33:30.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026163064, 2024-09-15T01:33:30.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:30.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:30.010 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:30.010 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:30.010 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:30.010 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 749 2024-09-15T01:33:30.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.010 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:30.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:30.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078047+0000", 2024-09-15T01:33:30.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:30.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:30.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078047+0000", 2024-09-15T01:33:30.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:30.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:30.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078048+0000", 2024-09-15T01:33:30.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2699999999999999e-07 2024-09-15T01:33:30.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:30.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078051+0000", 2024-09-15T01:33:30.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3289999999999998e-06 2024-09-15T01:33:30.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:30.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078060+0000", 2024-09-15T01:33:30.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3589999999999997e-06 2024-09-15T01:33:30.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:30.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078066+0000", 2024-09-15T01:33:30.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-15T01:33:30.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:30.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078241+0000", 2024-09-15T01:33:30.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017526299999999999 2024-09-15T01:33:30.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.078248+0000", 2024-09-15T01:33:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9489999999999999e-06 2024-09-15T01:33:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.080168+0000", 2024-09-15T01:33:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001919825 2024-09-15T01:33:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.080226+0000", 2024-09-15T01:33:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8428999999999999e-05 2024-09-15T01:33:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.103827+0000", 2024-09-15T01:33:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023600652 2024-09-15T01:33:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.104087+0000", 2024-09-15T01:33:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026007 2024-09-15T01:33:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-15T01:33:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:18.104210+0000", 2024-09-15T01:33:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012296400000000001 2024-09-15T01:33:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:30.016 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:30.017 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:33:30.019 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:33:30.020 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 reset_purged_snaps_last 2024-09-15T01:33:30.021 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:30.017+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:30.120 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:30.117+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:30.123 INFO:tasks.rados.rados.0.smithi175.stdout:325: done (6 left) 2024-09-15T01:33:30.123 INFO:tasks.rados.rados.0.smithi175.stdout:327: done (5 left) 2024-09-15T01:33:30.123 INFO:tasks.rados.rados.0.smithi175.stdout:330: done (4 left) 2024-09-15T01:33:30.123 INFO:tasks.rados.rados.0.smithi175.stdout:332: done (3 left) 2024-09-15T01:33:30.123 INFO:tasks.rados.rados.0.smithi175.stdout:333: done (2 left) 2024-09-15T01:33:30.123 INFO:tasks.rados.rados.0.smithi175.stdout:334: done (1 left) 2024-09-15T01:33:30.123 INFO:tasks.rados.rados.0.smithi175.stdout:335: done (0 left) 2024-09-15T01:33:30.123 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 43 current snap is 30 2024-09-15T01:33:30.123 INFO:tasks.rados.rados.0.smithi175.stdout:336: seq_num 143 ranges {3457024=458752,3915776=475136,4390912=655360,5046272=671744,5718016=147456} 2024-09-15T01:33:30.126 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:33:30.143 INFO:tasks.rados.rados.0.smithi175.stdout:336: writing smithi17529310-43 from 3457024 to 3915776 tid 1 2024-09-15T01:33:30.146 INFO:tasks.rados.rados.0.smithi175.stdout:336: writing smithi17529310-43 from 3915776 to 4390912 tid 2 2024-09-15T01:33:30.149 INFO:tasks.rados.rados.0.smithi175.stdout:336: writing smithi17529310-43 from 4390912 to 5046272 tid 3 2024-09-15T01:33:30.153 INFO:tasks.rados.rados.0.smithi175.stdout:336: writing smithi17529310-43 from 5046272 to 5718016 tid 4 2024-09-15T01:33:30.154 INFO:tasks.rados.rados.0.smithi175.stdout:336: writing smithi17529310-43 from 5718016 to 5865472 tid 5 2024-09-15T01:33:30.154 INFO:tasks.rados.rados.0.smithi175.stdout:337: setattr oid 47 current snap is 30 2024-09-15T01:33:30.154 INFO:tasks.rados.rados.0.smithi175.stdout:336: finishing write tid 1 to smithi17529310-43 2024-09-15T01:33:30.154 INFO:tasks.rados.rados.0.smithi175.stdout:336: finishing write tid 2 to smithi17529310-43 2024-09-15T01:33:30.154 INFO:tasks.rados.rados.0.smithi175.stdout:336: finishing write tid 3 to smithi17529310-43 2024-09-15T01:33:30.155 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 8 current snap is 30 2024-09-15T01:33:30.156 INFO:tasks.rados.rados.0.smithi175.stdout:338: seq_num 144 ranges {131072=16384} 2024-09-15T01:33:30.157 INFO:tasks.rados.rados.0.smithi175.stdout:338: writing smithi17529310-8 from 131072 to 147456 tid 1 2024-09-15T01:33:30.157 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 38 current snap is 30 2024-09-15T01:33:30.157 INFO:tasks.rados.rados.0.smithi175.stdout:339: seq_num 145 ranges {0=32768} 2024-09-15T01:33:30.157 INFO:tasks.rados.rados.0.smithi175.stdout:339: writing smithi17529310-38 from 0 to 32768 tid 1 2024-09-15T01:33:30.157 INFO:tasks.rados.rados.0.smithi175.stdout:340: rollback oid 17 current snap is 30 2024-09-15T01:33:30.157 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 17 to 25 2024-09-15T01:33:30.158 INFO:tasks.rados.rados.0.smithi175.stdout:341: snap_remove snap 27 2024-09-15T01:33:30.178 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:30.178 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-15T01:33:30.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.179 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4477.0:869 3.4s0 3:299c4018:::smithi17529310-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e88)", 2024-09-15T01:33:30.179 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-15T01:33:30.154942+0000", 2024-09-15T01:33:30.179 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022354755, 2024-09-15T01:33:30.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022381161, 2024-09-15T01:33:30.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-15T01:33:30.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-15T01:33:30.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-15T01:33:30.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4477", 2024-09-15T01:33:30.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:30.179 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 869 2024-09-15T01:33:30.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:30.154942+0000", 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:30.154942+0000", 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-15T01:33:30.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:30.154942+0000", 2024-09-15T01:33:30.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.06e-07 2024-09-15T01:33:30.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-15T01:33:30.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:30.154944+0000", 2024-09-15T01:33:30.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9089999999999998e-06 2024-09-15T01:33:30.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:30.154951+0000", 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4919999999999998e-06 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:30.154954+0000", 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7299999999999999e-06 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:30.160645+0000", 2024-09-15T01:33:30.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056904 2024-09-15T01:33:30.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:30.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:30.160744+0000", 2024-09-15T01:33:30.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8807000000000005e-05 2024-09-15T01:33:30.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-15T01:33:30.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:30.164388+0000", 2024-09-15T01:33:30.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003644146 2024-09-15T01:33:30.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-15T01:33:30.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-15T01:33:30.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-15T01:33:30.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-15T01:33:30.164490+0000", 2024-09-15T01:33:30.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101984 2024-09-15T01:33:30.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:30.184 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-15T01:33:30.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:30.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-15T01:33:30.184 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-15T01:33:30.184 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 1 2024-09-15T01:33:30.184 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:30.190 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:33:30.207 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:30.207 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:30.221 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:30.219+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:30.228 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 reset_purged_snaps_last 2024-09-15T01:33:30.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:30.248 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:30.249 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:30.270 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:30.270 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:30.291 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:30.291 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:30.298 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:30.298 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-15T01:33:30.298 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-15T01:33:30.298 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-15T01:33:30.298 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:30.308 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:33:30.310 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:30.311 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:30.322 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:30.320+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:30.330 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:30.330 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:30.349 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:30.350 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:30.372 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:30.372 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:30.391 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:30.392 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:30.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 reset_purged_snaps_last 2024-09-15T01:33:30.411 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:30.412 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:30.415 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:30.415 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:30.423 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:30.421+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:30.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-15T01:33:30.433 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:30.433 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:30.454 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:30.454 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:30.473 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:30.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:30.492+0000 7f50ae7fc640 0 -- 172.21.15.42:0/900780131 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7f50b0192890 msgr2=0x7f50b0194c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:30.499 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:30.520 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:30.520 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:30.523 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:30.522+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:30.540 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:30.540 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:30.549 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-15T01:33:30.641 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:30.640+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:30.699 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:30.699 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:30.707 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-15T01:33:30.725 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:30.724+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:30.826 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:30.825+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:30.848 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:30.848 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:30.857 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:33:30.927 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:30.926+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:30.967 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:30.967 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:30.976 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:33:31.028 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:31.026+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:31.075 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-15T01:33:31.079 INFO:tasks.rados.rados.0.smithi175.stdout:341: done (5 left) 2024-09-15T01:33:31.079 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 7 current snap is 30 2024-09-15T01:33:31.080 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 47 v 68 (ObjNum 33 snap 0 seq_num 1648604992) dirty exists 2024-09-15T01:33:31.080 INFO:tasks.rados.rados.0.smithi175.stdout:338: finishing write tid 1 to smithi17529310-8 2024-09-15T01:33:31.080 INFO:tasks.rados.rados.0.smithi175.stdout:338: finishing write tid 2 to smithi17529310-8 2024-09-15T01:33:31.080 INFO:tasks.rados.rados.0.smithi175.stdout:336: finishing write tid 4 to smithi17529310-43 2024-09-15T01:33:31.080 INFO:tasks.rados.rados.0.smithi175.stdout:339: finishing write tid 1 to smithi17529310-38 2024-09-15T01:33:31.080 INFO:tasks.rados.rados.0.smithi175.stdout:336: finishing write tid 5 to smithi17529310-43 2024-09-15T01:33:31.080 INFO:tasks.rados.rados.0.smithi175.stdout:339: finishing write tid 2 to smithi17529310-38 2024-09-15T01:33:31.080 INFO:tasks.rados.rados.0.smithi175.stdout:336: finishing write tid 6 to smithi17529310-43 2024-09-15T01:33:31.080 INFO:tasks.rados.rados.0.smithi175.stdout:342: seq_num 146 ranges {2654208=49152} 2024-09-15T01:33:31.095 INFO:tasks.rados.rados.0.smithi175.stdout:342: writing smithi17529310-7 from 2654208 to 2703360 tid 1 2024-09-15T01:33:31.095 INFO:tasks.rados.rados.0.smithi175.stdout:337: done (5 left) 2024-09-15T01:33:31.095 INFO:tasks.rados.rados.0.smithi175.stdout:343: snap_remove snap 24 2024-09-15T01:33:31.120 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:31.120 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:31.129 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:33:31.129 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:31.127+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:31.229 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:31.228+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:31.265 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:31.265 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:31.273 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:33:31.330 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:31.329+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:31.371 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:31.371 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:31.379 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:33:31.431 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:31.429+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:31.516 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:31.516 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:31.523 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:33:31.532 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:31.530+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:31.621 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:31.621 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:31.629 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:33:31.633 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:31.631+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:31.726 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:31.726 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:31.733 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:31.731+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:31.735 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:33:31.830 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:31.831 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:31.834 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:31.832+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:31.839 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:33:31.935 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:31.934+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:31.935 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:31.936 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:31.944 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-15T01:33:32.036 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:32.034+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:32.081 INFO:tasks.rados.rados.0.smithi175.stdout:343: done (5 left) 2024-09-15T01:33:32.081 INFO:tasks.rados.rados.0.smithi175.stdout:344: snap_create 2024-09-15T01:33:32.081 INFO:tasks.rados.rados.0.smithi175.stdout:338: finishing write tid 3 to smithi17529310-8 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 8 v 72 (ObjNum 144 snap 30 seq_num 144) dirty exists 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:338: left oid 8 (ObjNum 144 snap 30 seq_num 144) 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:340: finishing rollback tid 0 to smithi17529310-17 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:340: finishing rollback tid 1 to smithi17529310-17 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:340: finishing rollback tid 2 to smithi17529310-17 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 17 v 204 (ObjNum 126 snap 20 seq_num 126) dirty exists 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:339: finishing write tid 3 to smithi17529310-38 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 38 v 200 (ObjNum 145 snap 30 seq_num 145) dirty exists 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:339: left oid 38 (ObjNum 145 snap 30 seq_num 145) 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:336: finishing write tid 7 to smithi17529310-43 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 43 v 81 (ObjNum 143 snap 30 seq_num 143) dirty exists 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:336: left oid 43 (ObjNum 143 snap 30 seq_num 143) 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:342: finishing write tid 1 to smithi17529310-7 2024-09-15T01:33:32.082 INFO:tasks.rados.rados.0.smithi175.stdout:342: finishing write tid 2 to smithi17529310-7 2024-09-15T01:33:32.083 INFO:tasks.rados.rados.0.smithi175.stdout:342: finishing write tid 3 to smithi17529310-7 2024-09-15T01:33:32.083 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 7 v 71 (ObjNum 146 snap 30 seq_num 146) dirty exists 2024-09-15T01:33:32.083 INFO:tasks.rados.rados.0.smithi175.stdout:342: left oid 7 (ObjNum 146 snap 30 seq_num 146) 2024-09-15T01:33:32.083 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:32.083 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:32.095 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-15T01:33:32.138 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:32.137+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:32.191 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:32.192 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:32.200 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-15T01:33:32.237 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:32.236+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:32.337 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:32.337 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:32.338 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:32.336+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:32.346 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:33:32.438 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:32.437+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:32.453 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:32.453 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:32.462 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:33:32.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:33:32.539 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:32.538+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:32.559 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:32.559 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:32.567 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:33:32.641 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:32.639+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:32.664 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:32.665 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:32.673 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:33:32.674 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:32.694 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:32.694 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:32.713 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:32.713 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:32.733 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:32.733 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:32.742 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:32.741+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:32.751 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:32.752 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:32.770 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:32.770 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:32.771 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:32.771 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:32.779 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:33:32.789 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:32.790 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:32.809 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:32.809 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:32.827 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:32.828 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:32.843 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:32.840+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:32.847 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:32.847 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:32.866 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:32.866 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:32.876 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:32.876 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:32.876 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:32.876 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:32.876 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:32.884 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:33:32.886 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:32.887 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:32.905 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:32.905 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:32.924 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:32.924 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:32.943 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:32.944 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:32.944 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:32.943+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:32.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:32.942+0000 7f2f5e4ba640 0 -- 172.21.15.42:0/3231730545 >> [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] conn(0x7f2f58195c60 msgr2=0x7f2f581980c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-15T01:33:33.023 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.779677775000003, 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022664864999999999, 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.16e-07 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241155+0000", 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016715400000000001 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241334+0000", 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017883600000000001 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241342+0000", 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1519999999999992e-06 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241446+0000", 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010419 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241468+0000", 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1747e-05 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.244959+0000", 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0034911600000000001 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.244970+0000", 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1131000000000001e-05 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245013+0000", 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3214000000000003e-05 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245498+0000", 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00048501499999999999 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263652+0000", 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018154050000000001 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.779479815000002, 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022656372000000001, 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.029 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9399999999999999e-07 2024-09-15T01:33:33.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241203+0000", 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7152000000000001e-05 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241707+0000", 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050421299999999999 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241714+0000", 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4409999999999998e-06 2024-09-15T01:33:33.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241773+0000", 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9762000000000001e-05 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241783+0000", 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6860000000000001e-06 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245035+0000", 2024-09-15T01:33:33.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032521130000000001 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245184+0000", 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014925800000000001 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245769+0000", 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058491399999999996 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246140+0000", 2024-09-15T01:33:33.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037060199999999999 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263842+0000", 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.017702037 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.778262697000002, 2024-09-15T01:33:33.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034719353000000001, 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1600000000000001e-07 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242427+0000", 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4022000000000001e-05 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242460+0000", 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2496000000000002e-05 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242476+0000", 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6503000000000001e-05 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245083+0000", 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.002606531 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245092+0000", 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7679999999999997e-06 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245298+0000", 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000205971 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245772+0000", 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00047452200000000001 2024-09-15T01:33:33.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245817+0000", 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4474999999999997e-05 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246171+0000", 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035433200000000001 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277122+0000", 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030951117 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.777639166, 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031640535999999997, 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:33.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:33.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:33.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.49e-07 2024-09-15T01:33:33.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243044+0000", 2024-09-15T01:33:33.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7946999999999999e-05 2024-09-15T01:33:33.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243140+0000", 2024-09-15T01:33:33.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5311999999999998e-05 2024-09-15T01:33:33.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243147+0000", 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2069999999999998e-06 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245010+0000", 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018629230000000001 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245020+0000", 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0774999999999999e-05 2024-09-15T01:33:33.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245472+0000", 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045200900000000002 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245773+0000", 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00030029999999999998 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245817+0000", 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4367999999999999e-05 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246317+0000", 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050021500000000003 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274667+0000", 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028349330999999998 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.045 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.773375119999997, 2024-09-15T01:33:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027771575999999999, 2024-09-15T01:33:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247291+0000", 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7700000000000004e-07 2024-09-15T01:33:33.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247302+0000", 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1246e-05 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247333+0000", 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1179000000000001e-05 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247341+0000", 2024-09-15T01:33:33.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.571e-06 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247388+0000", 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6876999999999999e-05 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247395+0000", 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3359999999999997e-06 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247478+0000", 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2860000000000003e-05 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247487+0000", 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9029999999999993e-06 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247526+0000", 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8829999999999999e-05 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247546+0000", 2024-09-15T01:33:33.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0551999999999999e-05 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275062+0000", 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027515444999999999 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as5 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.7731523, 2024-09-15T01:33:33.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027915809999999999, 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247514+0000", 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9899999999999999e-07 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247524+0000", 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0396e-05 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247555+0000", 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0414000000000001e-05 2024-09-15T01:33:33.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247564+0000", 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1900000000000001e-06 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247616+0000", 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1788e-05 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247628+0000", 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.288e-05 2024-09-15T01:33:33.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247763+0000", 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013473700000000001 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247772+0000", 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3429999999999996e-06 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247804+0000", 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2533999999999997e-05 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247817+0000", 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2694000000000001e-05 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275429+0000", 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027612035 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s5 44/31 ECSubWrite(tid=65, reqid=osd.14.0:65, at_version=44'26, trim_to=0'0, roll_forward_to=44'24))", 2024-09-15T01:33:33.056 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.771106953999997, 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.026123550999999998, 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249559+0000", 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3200000000000003e-07 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249584+0000", 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4284999999999999e-05 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249633+0000", 2024-09-15T01:33:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9526e-05 2024-09-15T01:33:33.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249641+0000", 2024-09-15T01:33:33.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2239999999999995e-06 2024-09-15T01:33:33.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249695+0000", 2024-09-15T01:33:33.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3674999999999997e-05 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249705+0000", 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8300000000000008e-06 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251518+0000", 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018128409999999999 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251549+0000", 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0948999999999998e-05 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251591+0000", 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2515999999999997e-05 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251612+0000", 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0494000000000001e-05 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275682+0000", 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024070279 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s5 44/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=44'26))", 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.767913243999999, 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022959355000000001, 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252753+0000", 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7499999999999998e-07 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252764+0000", 2024-09-15T01:33:33.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1123e-05 2024-09-15T01:33:33.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252899+0000", 2024-09-15T01:33:33.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013444100000000001 2024-09-15T01:33:33.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252906+0000", 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8650000000000001e-06 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252959+0000", 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2698000000000003e-05 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252968+0000", 2024-09-15T01:33:33.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2140000000000002e-06 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253058+0000", 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9926999999999995e-05 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253066+0000", 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6620000000000005e-06 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253105+0000", 2024-09-15T01:33:33.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8875999999999998e-05 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253117+0000", 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2405000000000001e-05 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275711+0000", 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022594169000000001 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=60, reqid=client.4477.0:306, at_version=44'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.749836844000001, 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024129333999999999, 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270829+0000", 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270845+0000", 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6013999999999999e-05 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270871+0000", 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6140999999999999e-05 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270877+0000", 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7989999999999999e-06 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270947+0000", 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9480000000000006e-05 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270959+0000", 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2395000000000001e-05 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274484+0000", 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003524976 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274715+0000", 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000230595 2024-09-15T01:33:33.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274806+0000", 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1130000000000003e-05 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274847+0000", 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1473000000000001e-05 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.294958+0000", 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020110711 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.749758833999998, 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020359914999999999, 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270907+0000", 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4200000000000001e-07 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270919+0000", 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1761e-05 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270936+0000", 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7761000000000002e-05 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270953+0000", 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6858e-05 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272042+0000", 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010882769999999999 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272051+0000", 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8400000000000007e-06 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272145+0000", 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3639000000000003e-05 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272153+0000", 2024-09-15T01:33:33.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6240000000000002e-06 2024-09-15T01:33:33.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272221+0000", 2024-09-15T01:33:33.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8281999999999994e-05 2024-09-15T01:33:33.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272241+0000", 2024-09-15T01:33:33.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9542e-05 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.291266+0000", 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019025889000000001 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.748247587999998, 2024-09-15T01:33:33.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.025858174000000001, 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272446+0000", 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7114000000000002e-05 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272489+0000", 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3711999999999998e-05 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272498+0000", 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9169999999999995e-06 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274537+0000", 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0020384190000000001 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274545+0000", 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1999999999999994e-06 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274659+0000", 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011420499999999999 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274716+0000", 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7114999999999998e-05 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274807+0000", 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0346999999999994e-05 2024-09-15T01:33:33.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274883+0000", 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.695e-05 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298276+0000", 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023392452000000001 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.084 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=62, reqid=client.4477.0:308, at_version=44'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:33.084 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:33.084 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.747200438, 2024-09-15T01:33:33.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024943603000000002, 2024-09-15T01:33:33.084 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.084 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.084 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.084 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:33.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.37e-07 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273478+0000", 2024-09-15T01:33:33.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2768e-05 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273502+0000", 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3920000000000001e-05 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273508+0000", 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0909999999999998e-06 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274699+0000", 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011909240000000001 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274707+0000", 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5429999999999996e-06 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274788+0000", 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1804000000000005e-05 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274811+0000", 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2379e-05 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274902+0000", 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1031000000000003e-05 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274910+0000", 2024-09-15T01:33:33.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8359999999999992e-06 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298408+0000", 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023498769999999999 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=92, reqid=client.4477.0:309, at_version=44'31, trim_to=0'0, roll_forward_to=44'28))", 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.74610096, 2024-09-15T01:33:33.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024021724000000001, 2024-09-15T01:33:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274565+0000", 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9299999999999996e-07 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275016+0000", 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045075999999999997 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275077+0000", 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0582999999999999e-05 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275084+0000", 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1289999999999998e-06 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275144+0000", 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0566000000000003e-05 2024-09-15T01:33:33.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275153+0000", 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4849999999999992e-06 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275302+0000", 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000149689 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275327+0000", 2024-09-15T01:33:33.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4337000000000001e-05 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277020+0000", 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001693153 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277074+0000", 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4122999999999997e-05 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298586+0000", 2024-09-15T01:33:33.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021512006 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 44/31 ECSubWrite(tid=35, reqid=client.4477.0:296, at_version=0'0, trim_to=0'0, roll_forward_to=44'25))", 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.742558985999999, 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020797596000000002, 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278107+0000", 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8199999999999998e-07 2024-09-15T01:33:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278119+0000", 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1567e-05 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278155+0000", 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6560999999999998e-05 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278163+0000", 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2819999999999999e-06 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278209+0000", 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6146999999999997e-05 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278219+0000", 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0699999999999999e-05 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278342+0000", 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000122061 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278350+0000", 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4460000000000001e-06 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280088+0000", 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0017376309999999999 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280107+0000", 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.986e-05 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298904+0000", 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018796459000000001 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=93, reqid=client.4477.0:310, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:33.100 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.741850982999999, 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020226037999999998, 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:33.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278815+0000", 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6000000000000003e-07 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279204+0000", 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000388413 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279228+0000", 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4674000000000001e-05 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279233+0000", 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7849999999999999e-06 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279272+0000", 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9502999999999998e-05 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279278+0000", 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0079999999999999e-06 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279401+0000", 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012238799999999999 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280091+0000", 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00069018900000000002 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280213+0000", 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012227700000000001 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280231+0000", 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7408999999999999e-05 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.299040+0000", 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018809632 2024-09-15T01:33:33.105 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.5s6 44/31 ECSubWrite(tid=15, reqid=client.4477.0:314, at_version=44'8, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.741301526999997, 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.026604415999999999, 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.106 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279386+0000", 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.211e-05 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279421+0000", 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4523000000000001e-05 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279428+0000", 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4420000000000001e-06 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279474+0000", 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5995000000000003e-05 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279480+0000", 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.105e-06 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279645+0000", 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016430199999999999 2024-09-15T01:33:33.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280092+0000", 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044707999999999998 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280214+0000", 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012248399999999999 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280260+0000", 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5785000000000003e-05 2024-09-15T01:33:33.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.305968+0000", 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.025708106000000001 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.5s6 44/31 ECSubWrite(tid=16, reqid=client.4477.0:314, at_version=0'0, trim_to=0'0, roll_forward_to=44'8))", 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.738646953, 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024148095000000001, 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.111 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282019+0000", 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282028+0000", 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0969999999999995e-06 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282054+0000", 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5897e-05 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282059+0000", 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7849999999999999e-06 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282101+0000", 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1801000000000002e-05 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282108+0000", 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0650000000000001e-06 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282171+0000", 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2834000000000002e-05 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282844+0000", 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00067343600000000002 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282915+0000", 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1242999999999995e-05 2024-09-15T01:33:33.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282933+0000", 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8136000000000001e-05 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306166+0000", 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023233052000000001 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=63, reqid=client.4477.0:308, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.737063321000001, 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022584495, 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283603+0000", 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5899999999999995e-07 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283618+0000", 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5600999999999999e-05 2024-09-15T01:33:33.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283648+0000", 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9295e-05 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283653+0000", 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1050000000000003e-06 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283703+0000", 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9883999999999999e-05 2024-09-15T01:33:33.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283712+0000", 2024-09-15T01:33:33.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3740000000000004e-06 2024-09-15T01:33:33.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283859+0000", 2024-09-15T01:33:33.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000146959 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283869+0000", 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0295999999999999e-05 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283906+0000", 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6534e-05 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283922+0000", 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6560000000000001e-05 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306187+0000", 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022264128000000001 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 47/38 ECSubWrite(tid=76, reqid=client.4477.0:376, at_version=47'25, trim_to=0'0, roll_forward_to=47'22))", 2024-09-15T01:33:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 43.656859158000003, 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.61852187999999997, 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:33.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363807+0000", 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5799999999999999e-07 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363818+0000", 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1729000000000001e-05 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981818+0000", 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.61799953399999996 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981826+0000", 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1049999999999995e-06 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981893+0000", 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7229000000000003e-05 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981905+0000", 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1527e-05 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982064+0000", 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000159147 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982078+0000", 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:33:33.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982122+0000", 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4354000000000003e-05 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982140+0000", 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8315e-05 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982328+0000", 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018781200000000001 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.127 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 54/52 ECSubWrite(tid=119, reqid=client.4477.0:500, at_version=54'41, trim_to=0'0, roll_forward_to=54'38))", 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 36.809364127000002, 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.27782579699999999, 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:33.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211302+0000", 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.89e-07 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211313+0000", 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1521e-05 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488807+0000", 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.27749379200000002 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488811+0000", 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2889999999999999e-06 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488867+0000", 2024-09-15T01:33:33.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5322000000000003e-05 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488874+0000", 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9659999999999996e-06 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488953+0000", 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8591999999999996e-05 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488959+0000", 2024-09-15T01:33:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0000000000000002e-06 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488993+0000", 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4326999999999998e-05 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.489009+0000", 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6015000000000002e-05 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.489127+0000", 2024-09-15T01:33:33.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117584 2024-09-15T01:33:33.133 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.133 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.133 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.133 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.133 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.133 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:33.133 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T01:33:33.135 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:33.133+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:33.136 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:33:33.137 INFO:tasks.rados.rados.0.smithi175.stdout:336: done (5 left) 2024-09-15T01:33:33.137 INFO:tasks.rados.rados.0.smithi175.stdout:338: done (4 left) 2024-09-15T01:33:33.137 INFO:tasks.rados.rados.0.smithi175.stdout:339: done (3 left) 2024-09-15T01:33:33.137 INFO:tasks.rados.rados.0.smithi175.stdout:340: done (2 left) 2024-09-15T01:33:33.137 INFO:tasks.rados.rados.0.smithi175.stdout:342: done (1 left) 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:344: done (0 left) 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:345: read oid 18 snap -1 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:345: expect (ObjNum 115 snap 18 seq_num 115) 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 38 current snap is 31 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:346: seq_num 147 ranges {32768=65536} 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:346: writing smithi17529310-38 from 32768 to 98304 tid 1 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:347: read oid 17 snap -1 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:347: expect (ObjNum 126 snap 20 seq_num 126) 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:348: read oid 42 snap -1 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:348: expect (ObjNum 105 snap 10 seq_num 105) 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:349: read oid 36 snap 12 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:349: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:350: read oid 26 snap 29 2024-09-15T01:33:33.138 INFO:tasks.rados.rados.0.smithi175.stdout:350: expect (ObjNum 103 snap 10 seq_num 103) 2024-09-15T01:33:33.139 INFO:tasks.rados.rados.0.smithi175.stdout:351: delete oid 25 current snap is 31 2024-09-15T01:33:33.139 INFO:tasks.rados.rados.0.smithi175.stdout:351: done (6 left) 2024-09-15T01:33:33.139 INFO:tasks.rados.rados.0.smithi175.stdout:352: snap_remove snap 30 2024-09-15T01:33:33.139 INFO:tasks.rados.rados.0.smithi175.stdout:346: finishing write tid 1 to smithi17529310-38 2024-09-15T01:33:33.164 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:33.164 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:33.235 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:33.235 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:33.235 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:33.235 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:33.236 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:33.234+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:33.244 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:33:33.336 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:33.335+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:33.341 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:33.341 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:33.341 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:33.341 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:33.341 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:33.349 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:33:33.437 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:33.436+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:33.448 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.198461739999999, 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011534543, 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246894+0000", 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0500000000000002e-07 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2585000000000001e-05 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246951+0000", 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4552999999999999e-05 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246961+0000", 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0516000000000001e-05 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247025+0000", 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3540999999999994e-05 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247039+0000", 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4484000000000001e-05 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247338+0000", 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00029853699999999998 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248324+0000", 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0009866059999999999 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248375+0000", 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0337000000000003e-05 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248387+0000", 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2081e-05 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258427+0000", 2024-09-15T01:33:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010040498 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as6 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.197817600999997, 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010349413, 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247538+0000", 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0130000000000001e-06 2024-09-15T01:33:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247552+0000", 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3723999999999999e-05 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247588+0000", 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5989999999999999e-05 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247595+0000", 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5789999999999998e-06 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247648+0000", 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2757000000000003e-05 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247654+0000", 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3629999999999999e-06 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247737+0000", 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2886000000000006e-05 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248326+0000", 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058858900000000004 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248380+0000", 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3832999999999998e-05 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248401+0000", 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0843e-05 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257886+0000", 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0094858359999999992 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.186286246999998, 2024-09-15T01:33:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015398044, 2024-09-15T01:33:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259069+0000", 2024-09-15T01:33:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2399999999999996e-07 2024-09-15T01:33:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259095+0000", 2024-09-15T01:33:33.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5527000000000001e-05 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259133+0000", 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8211999999999999e-05 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259140+0000", 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9330000000000004e-06 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259189+0000", 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8797000000000001e-05 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259200+0000", 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0932e-05 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259425+0000", 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000225505 2024-09-15T01:33:33.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259453+0000", 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8152000000000001e-05 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.261606+0000", 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021525870000000001 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.261621+0000", 2024-09-15T01:33:33.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4829e-05 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274466+0000", 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012845646 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 44/31 ECSubWrite(tid=39, reqid=client.4477.0:283, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:33.465 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.176962037999999, 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014946894, 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0299999999999998e-07 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268593+0000", 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000199278 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268633+0000", 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0367000000000002e-05 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268640+0000", 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5100000000000001e-06 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268742+0000", 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010184899999999999 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268752+0000", 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9299999999999998e-06 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269007+0000", 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000255016 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269035+0000", 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8039999999999999e-05 2024-09-15T01:33:33.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271592+0000", 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025566260000000002 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271610+0000", 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7629999999999999e-05 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283339+0000", 2024-09-15T01:33:33.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011729844999999999 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.6s6 44/38 ECSubWrite(tid=22, reqid=client.4477.0:293, at_version=44'14, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.174178517000001, 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012212746, 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271177+0000", 2024-09-15T01:33:33.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271394+0000", 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021719400000000001 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271421+0000", 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6842000000000002e-05 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271426+0000", 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9010000000000002e-06 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271464+0000", 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7928e-05 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271476+0000", 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1979e-05 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271800+0000", 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032464100000000002 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271832+0000", 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1501999999999998e-05 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271921+0000", 2024-09-15T01:33:33.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9514999999999999e-05 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271940+0000", 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8436000000000001e-05 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283389+0000", 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011449086000000001 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.476 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 55/31 ECSubWrite(tid=160, reqid=client.4477.0:512, at_version=55'53, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 36.369677842999998, 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011287287999999999, 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:33.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075698+0000", 2024-09-15T01:33:33.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0384999999999999e-05 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075734+0000", 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6300999999999997e-05 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075743+0000", 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9549999999999998e-06 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075803+0000", 2024-09-15T01:33:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0304999999999999e-05 2024-09-15T01:33:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075813+0000", 2024-09-15T01:33:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9539999999999999e-06 2024-09-15T01:33:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075916+0000", 2024-09-15T01:33:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102534 2024-09-15T01:33:33.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075925+0000", 2024-09-15T01:33:33.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1509999999999993e-06 2024-09-15T01:33:33.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075971+0000", 2024-09-15T01:33:33.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6276999999999998e-05 2024-09-15T01:33:33.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075995+0000", 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3176999999999998e-05 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.086964+0000", 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010969504999999999 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 55/31 ECSubWrite(tid=161, reqid=client.4477.0:513, at_version=55'54, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:33.482 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 36.369041398999997, 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010838802, 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076314+0000", 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076329+0000", 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5505000000000001e-05 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076356+0000", 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6324000000000001e-05 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076362+0000", 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2820000000000002e-06 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076409+0000", 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7190000000000001e-05 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076419+0000", 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9450000000000005e-06 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076491+0000", 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1680999999999994e-05 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076497+0000", 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3810000000000004e-06 2024-09-15T01:33:33.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076958+0000", 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00046042499999999998 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076971+0000", 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2999999999999999e-05 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.087152+0000", 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010181366000000001 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 59/31 ECSubWrite(tid=179, reqid=client.4477.0:552, at_version=59'58, trim_to=0'0, roll_forward_to=56'56))", 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 32.353872129999999, 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010784815, 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.488 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091483+0000", 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2999999999999999e-07 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091650+0000", 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016715 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091673+0000", 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3434999999999998e-05 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091680+0000", 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7329999999999997e-06 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091721+0000", 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1088999999999999e-05 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091729+0000", 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9170000000000006e-06 2024-09-15T01:33:33.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091940+0000", 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021130000000000001 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.092580+0000", 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00063981200000000002 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093157+0000", 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057673800000000003 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093178+0000", 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0911e-05 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.102267+0000", 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090895000000000004 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.493 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 66/31 ECSubWrite(tid=64, reqid=client.4477.0:609, at_version=66'56, trim_to=0'0, roll_forward_to=66'52))", 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:08.087598+0000", 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 25.357756623, 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010297796999999999, 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.087598+0000", 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.087598+0000", 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.087599+0000", 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1100000000000005e-07 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088049+0000", 2024-09-15T01:33:33.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045032600000000002 2024-09-15T01:33:33.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088087+0000", 2024-09-15T01:33:33.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8138e-05 2024-09-15T01:33:33.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088095+0000", 2024-09-15T01:33:33.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.34e-06 2024-09-15T01:33:33.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088153+0000", 2024-09-15T01:33:33.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-05 2024-09-15T01:33:33.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088165+0000", 2024-09-15T01:33:33.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1386000000000001e-05 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088485+0000", 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032002700000000002 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088493+0000", 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8550000000000008e-06 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088596+0000", 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102278 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088613+0000", 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7757e-05 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.097896+0000", 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0092822389999999994 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 71/31 ECSubWrite(tid=229, reqid=client.4477.0:651, at_version=71'127, trim_to=0'0, roll_forward_to=71'124))", 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.208012+0000", 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.237342218999999, 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.016322817, 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208012+0000", 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208012+0000", 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208013+0000", 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3099999999999997e-07 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208033+0000", 2024-09-15T01:33:33.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9539e-05 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208064+0000", 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1705999999999998e-05 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208073+0000", 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3750000000000003e-06 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208136+0000", 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3736000000000001e-05 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208146+0000", 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9029999999999999e-06 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208264+0000", 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117372 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208272+0000", 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.755e-06 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208300+0000", 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7875999999999999e-05 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208320+0000", 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9791000000000001e-05 2024-09-15T01:33:33.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224335+0000", 2024-09-15T01:33:33.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.016015033000000001 2024-09-15T01:33:33.505 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.505 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.505 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.505 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 71/31 ECSubWrite(tid=231, reqid=client.4477.0:652, at_version=71'128, trim_to=0'0, roll_forward_to=71'124))", 2024-09-15T01:33:33.505 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.209071+0000", 2024-09-15T01:33:33.505 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.236283442000001, 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015677593, 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209071+0000", 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209071+0000", 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209072+0000", 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1399999999999997e-07 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209084+0000", 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1917e-05 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209402+0000", 2024-09-15T01:33:33.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00031803499999999999 2024-09-15T01:33:33.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209407+0000", 2024-09-15T01:33:33.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3179999999999997e-06 2024-09-15T01:33:33.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209454+0000", 2024-09-15T01:33:33.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6981000000000003e-05 2024-09-15T01:33:33.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209464+0000", 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0037999999999999e-05 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209592+0000", 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012783 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209599+0000", 2024-09-15T01:33:33.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9489999999999999e-06 2024-09-15T01:33:33.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209779+0000", 2024-09-15T01:33:33.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017987 2024-09-15T01:33:33.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209797+0000", 2024-09-15T01:33:33.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8071e-05 2024-09-15T01:33:33.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224749+0000", 2024-09-15T01:33:33.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.01495207 2024-09-15T01:33:33.511 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.511 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.511 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.511 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 71/31 ECSubWrite(tid=232, reqid=client.4477.0:652, at_version=0'0, trim_to=0'0, roll_forward_to=71'128))", 2024-09-15T01:33:33.511 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:33.511 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.234873846999999, 2024-09-15T01:33:33.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014733626, 2024-09-15T01:33:33.512 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.512 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.512 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:33.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:33.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:33.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8999999999999996e-07 2024-09-15T01:33:33.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210491+0000", 2024-09-15T01:33:33.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7669999999999998e-06 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210530+0000", 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9039999999999999e-05 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210538+0000", 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2029999999999992e-06 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210601+0000", 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3069000000000002e-05 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210612+0000", 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0372e-05 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210761+0000", 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000149044 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210833+0000", 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2084999999999998e-05 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210854+0000", 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1424999999999999e-05 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210863+0000", 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1430000000000004e-06 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.225214+0000", 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014350887999999999 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=233, reqid=client.4477.0:653, at_version=0'0, trim_to=0'0, roll_forward_to=71'67))", 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.234860053999999, 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013895739000000001, 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.517 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:33.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0599999999999999e-07 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210501+0000", 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2310000000000001e-06 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210562+0000", 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0612000000000002e-05 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210570+0000", 2024-09-15T01:33:33.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4109999999999996e-06 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210619+0000", 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9419000000000002e-05 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210625+0000", 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5339999999999999e-06 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210691+0000", 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5905e-05 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210698+0000", 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0910000000000004e-06 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210831+0000", 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000133065 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210844+0000", 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3349e-05 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224390+0000", 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013545916 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=234, reqid=client.4477.0:654, at_version=71'68, trim_to=0'0, roll_forward_to=71'67))", 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:33.522 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.23370963, 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013353726, 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2900000000000004e-07 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212326+0000", 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00068024600000000002 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212354+0000", 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8555999999999999e-05 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212360+0000", 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.755e-06 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212397+0000", 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7086999999999999e-05 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212404+0000", 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7190000000000004e-06 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212510+0000", 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010600800000000001 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212515+0000", 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3159999999999996e-06 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213020+0000", 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000504854 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213032+0000", 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2048999999999999e-05 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224999+0000", 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011966607000000001 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.527 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=235, reqid=client.4477.0:654, at_version=0'0, trim_to=0'0, roll_forward_to=71'68))", 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.231802798, 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011535082, 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4799999999999998e-07 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213560+0000", 2024-09-15T01:33:33.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213582+0000", 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2524e-05 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213588+0000", 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9009999999999999e-06 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213624+0000", 2024-09-15T01:33:33.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5725000000000001e-05 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213629+0000", 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0679999999999996e-06 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213684+0000", 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4722999999999998e-05 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213690+0000", 2024-09-15T01:33:33.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4489999999999996e-06 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.215563+0000", 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018726680000000001 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.215576+0000", 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3021e-05 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.225087+0000", 2024-09-15T01:33:33.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0095110449999999992 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 85/31 ECSubWrite(tid=351, reqid=osd.11.0:351, at_version=85'111, trim_to=0'0, roll_forward_to=85'107))", 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:27.141013+0000", 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 6.3043420030000004, 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.109127261, 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.141013+0000", 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.141013+0000", 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.141013+0000", 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.06e-07 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.141037+0000", 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4209000000000001e-05 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249578+0000", 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.10854032600000001 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249589+0000", 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1396000000000001e-05 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249676+0000", 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6737999999999995e-05 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249682+0000", 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0859999999999998e-06 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249864+0000", 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018231000000000001 2024-09-15T01:33:33.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249872+0000", 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9370000000000004e-06 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249905+0000", 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2852000000000003e-05 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249918+0000", 2024-09-15T01:33:33.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2972e-05 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.250140+0000", 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022172899999999999 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 88/31 ECSubWrite(tid=377, reqid=client.4477.0:877, at_version=88'202, trim_to=0'0, roll_forward_to=87'198))", 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:30.160526+0000", 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.284829019, 2024-09-15T01:33:33.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015647682999999999, 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160526+0000", 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160526+0000", 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160526+0000", 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4099999999999997e-07 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160546+0000", 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9592000000000001e-05 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160582+0000", 2024-09-15T01:33:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5706e-05 2024-09-15T01:33:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160588+0000", 2024-09-15T01:33:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7399999999999998e-06 2024-09-15T01:33:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160647+0000", 2024-09-15T01:33:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8125999999999999e-05 2024-09-15T01:33:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160655+0000", 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4470000000000006e-06 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160917+0000", 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00026217299999999999 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160924+0000", 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2609999999999996e-06 2024-09-15T01:33:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163192+0000", 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022676559999999998 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163210+0000", 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7781000000000001e-05 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.176173+0000", 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.01296336 2024-09-15T01:33:33.543 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 88/31 ECSubWrite(tid=379, reqid=client.4477.0:878, at_version=88'203, trim_to=0'0, roll_forward_to=87'198))", 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:30.161562+0000", 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.2837923299999998, 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.016881072, 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161562+0000", 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161562+0000", 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161563+0000", 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1399999999999996e-07 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161579+0000", 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.641e-05 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161619+0000", 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9190999999999998e-05 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161626+0000", 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6109999999999996e-06 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161673+0000", 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6524999999999999e-05 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161682+0000", 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5799999999999998e-06 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161833+0000", 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015063199999999999 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163196+0000", 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0013626339999999999 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163354+0000", 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015873699999999999 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163372+0000", 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7430000000000001e-05 2024-09-15T01:33:33.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.178443+0000", 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015071508000000001 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 88/31 ECSubWrite(tid=381, reqid=client.4477.0:879, at_version=88'204, trim_to=0'0, roll_forward_to=88'199))", 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:30.162496+0000", 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.2828581890000001, 2024-09-15T01:33:33.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.01606695, 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162496+0000", 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162496+0000", 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162497+0000", 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8599999999999997e-07 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162517+0000", 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9426e-05 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162545+0000", 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8138999999999999e-05 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162551+0000", 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0970000000000002e-06 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162598+0000", 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7187000000000001e-05 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162611+0000", 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2505e-05 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162769+0000", 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015802899999999999 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163200+0000", 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043180900000000001 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163359+0000", 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015842699999999999 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163400+0000", 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1702000000000002e-05 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.178563+0000", 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015162843000000001 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 88/31 ECSubWrite(tid=383, reqid=client.4477.0:879, at_version=0'0, trim_to=0'0, roll_forward_to=88'204))", 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:30.164585+0000", 2024-09-15T01:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.2807691110000001, 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014151115000000001, 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164585+0000", 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164585+0000", 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164586+0000", 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3499999999999995e-07 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164598+0000", 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1935e-05 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164624+0000", 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5931e-05 2024-09-15T01:33:33.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164630+0000", 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6099999999999997e-06 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164685+0000", 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5337999999999998e-05 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164691+0000", 2024-09-15T01:33:33.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4999999999999996e-06 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164830+0000", 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013819900000000001 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164840+0000", 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9820000000000003e-06 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.166597+0000", 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001757273 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.166633+0000", 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6386e-05 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.178737+0000", 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012103226 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.559 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.560 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.560 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:33.561 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-15T01:33:33.562 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:33.560+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:33.658 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:33.658 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:33.658 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:33.659 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:33.662 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:33.659+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:33.667 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-15T01:33:33.763 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:33.761+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:33.804 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:33.804 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:33.804 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:33.804 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:33.804 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:33.812 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-15T01:33:33.864 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:33.862+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:33.911 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:33.911 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:33.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:33.911 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:33.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.911 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:74 3.0s0 3:0187269c:::smithi17529310-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:33.911 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:33.911 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 57.697771349999996, 2024-09-15T01:33:33.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.071208814999999995, 2024-09-15T01:33:33.911 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.911 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 74 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1200000000000003e-07 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210620+0000", 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.278e-06 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210623+0000", 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5779999999999999e-06 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210626+0000", 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.295e-06 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211745+0000", 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011191770000000001 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211880+0000", 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013497499999999999 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212353+0000", 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000473616 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212441+0000", 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7268999999999994e-05 2024-09-15T01:33:33.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212454+0000", 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3529999999999999e-05 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212483+0000", 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9224e-05 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212755+0000", 2024-09-15T01:33:33.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00027191599999999998 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.281770+0000", 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.069014438999999997 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.281826+0000", 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6005999999999999e-05 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:75 3.0s0 3:0187269c:::smithi17529310-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 57.697739110000001, 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.072451382999999994, 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 75 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4700000000000002e-07 2024-09-15T01:33:33.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210652+0000", 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9160000000000001e-06 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210657+0000", 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0529999999999997e-06 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210660+0000", 2024-09-15T01:33:33.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4339999999999999e-06 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212500+0000", 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001839732 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212584+0000", 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4759999999999995e-05 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.282036+0000", 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.069451322999999995 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.282131+0000", 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.577e-05 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.283100+0000", 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00096904800000000002 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds2 34/31 ECSubWrite(tid=16, reqid=client.4477.0:116, at_version=34'10, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 57.648764661000001, 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.24562283800000001, 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9599999999999999e-07 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259634+0000", 2024-09-15T01:33:33.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0412e-05 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.504838+0000", 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.245203848 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.504852+0000", 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3346999999999999e-05 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505000+0000", 2024-09-15T01:33:33.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000148209 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505006+0000", 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6309999999999997e-06 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505068+0000", 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1962000000000001e-05 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505074+0000", 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0579999999999997e-06 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505103+0000", 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9828e-05 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505116+0000", 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2261e-05 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505246+0000", 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000130786 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds2 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.661472013999997, 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033387655000000002, 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:33.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:33.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246917+0000", 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.61e-07 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246937+0000", 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9741999999999998e-05 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246964+0000", 2024-09-15T01:33:33.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7013999999999999e-05 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246970+0000", 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8250000000000001e-06 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247020+0000", 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0875999999999999e-05 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247032+0000", 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1280999999999999e-05 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249500+0000", 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0024682720000000001 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251106+0000", 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001606489 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251175+0000", 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8926999999999999e-05 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251258+0000", 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2441000000000006e-05 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280304+0000", 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029046026999999999 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.933 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs2 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247251+0000", 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.661137388, 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033110460000000001, 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247251+0000", 2024-09-15T01:33:33.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247251+0000", 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247252+0000", 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9899999999999999e-07 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247274+0000", 2024-09-15T01:33:33.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2160000000000001e-05 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247314+0000", 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0438000000000001e-05 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247321+0000", 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.187e-06 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249348+0000", 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.002026524 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249356+0000", 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7319999999999998e-06 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249538+0000", 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000182239 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251107+0000", 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001569094 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251176+0000", 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8707999999999993e-05 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251269+0000", 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3246999999999997e-05 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280361+0000", 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029092331999999999 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s5 44/31 ECSubWrite(tid=28, reqid=osd.8.0:28, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.250215+0000", 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.658173218000002, 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033811606000000001, 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.939 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250215+0000", 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250215+0000", 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250216+0000", 2024-09-15T01:33:33.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3200000000000003e-07 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250240+0000", 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3849000000000002e-05 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250277+0000", 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7469999999999999e-05 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250284+0000", 2024-09-15T01:33:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0979999999999996e-06 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250325+0000", 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0184e-05 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250331+0000", 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5060000000000001e-06 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254385+0000", 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0040539560000000001 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254397+0000", 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2502e-05 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254432+0000", 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4631000000000001e-05 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254987+0000", 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000555011 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.284027+0000", 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029039466999999999 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s5 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:33.944 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.251962+0000", 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.656425902999999, 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034678124999999997, 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251962+0000", 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251962+0000", 2024-09-15T01:33:33.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251963+0000", 2024-09-15T01:33:33.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4799999999999998e-07 2024-09-15T01:33:33.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251986+0000", 2024-09-15T01:33:33.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3221e-05 2024-09-15T01:33:33.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252099+0000", 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112687 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252107+0000", 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0399999999999993e-06 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254434+0000", 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0023271300000000002 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254447+0000", 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2607e-05 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254678+0000", 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000230869 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254847+0000", 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016962300000000001 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254891+0000", 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3441e-05 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.255014+0000", 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012353200000000001 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286640+0000", 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031626327000000003 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs2 44/31 ECSubWrite(tid=91, reqid=osd.11.0:87, at_version=0'0, trim_to=0'0, roll_forward_to=44'44))", 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267434+0000", 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.640954243000003, 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039301718999999999, 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.950 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267434+0000", 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267434+0000", 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267435+0000", 2024-09-15T01:33:33.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.92e-07 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267445+0000", 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8949999999999993e-06 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267477+0000", 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2768000000000002e-05 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267485+0000", 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4499999999999998e-06 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281136+0000", 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013651389999999999 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281144+0000", 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9549999999999992e-06 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.953 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281264+0000", 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012005799999999999 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282867+0000", 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00160271 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282917+0000", 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9919000000000001e-05 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282948+0000", 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1053000000000002e-05 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306736+0000", 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023787729000000001 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s6 44/38 ECSubWrite(tid=60, reqid=client.4477.0:306, at_version=44'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:33.955 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270872+0000", 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.637516181000002, 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039161770999999998, 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270872+0000", 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270872+0000", 2024-09-15T01:33:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270873+0000", 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0219999999999999e-06 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270902+0000", 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9079000000000001e-05 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270938+0000", 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5849999999999997e-05 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270958+0000", 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9762000000000002e-05 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280987+0000", 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010029733000000001 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280996+0000", 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0280000000000006e-06 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281214+0000", 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021798100000000001 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282866+0000", 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001651292 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.959 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282916+0000", 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0145000000000002e-05 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282941+0000", 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5066e-05 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.310034+0000", 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027092813 2024-09-15T01:33:33.960 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s4 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270969+0000", 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.637419012000002, 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034685404000000003, 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.961 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270969+0000", 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270969+0000", 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270970+0000", 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6199999999999998e-07 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270986+0000", 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6622000000000001e-05 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271019+0000", 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3077000000000002e-05 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271025+0000", 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8699999999999997e-06 2024-09-15T01:33:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274545+0000", 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0035193350000000002 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274553+0000", 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2810000000000008e-06 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274633+0000", 2024-09-15T01:33:33.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0358000000000005e-05 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274827+0000", 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019425000000000001 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.276977+0000", 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021494420000000001 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.276994+0000", 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6733000000000001e-05 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.305655+0000", 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028660973999999999 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s4 44/31 ECSubWrite(tid=34, reqid=client.4477.0:296, at_version=44'25, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.636905699000003, 2024-09-15T01:33:33.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034642487, 2024-09-15T01:33:33.967 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.967 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.967 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.967 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.967 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:33.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.967 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.967 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.967 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2600000000000002e-07 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271500+0000", 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6745999999999999e-05 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271570+0000", 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0242999999999998e-05 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271578+0000", 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0870000000000007e-06 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274674+0000", 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030959379999999999 2024-09-15T01:33:33.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274682+0000", 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9570000000000002e-06 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274766+0000", 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3630999999999999e-05 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274828+0000", 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2609999999999999e-05 2024-09-15T01:33:33.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.276978+0000", 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021496520000000002 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277024+0000", 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5723999999999998e-05 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306125+0000", 2024-09-15T01:33:33.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029101373 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s6 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.272445+0000", 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.635942882000002, 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041201330000000001, 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272445+0000", 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272445+0000", 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272446+0000", 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8299999999999995e-07 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.973 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272467+0000", 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0319999999999999e-05 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272505+0000", 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8862999999999997e-05 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272513+0000", 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.362e-06 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281243+0000", 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0087304259999999995 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281250+0000", 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9399999999999996e-06 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281315+0000", 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4664000000000003e-05 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282868+0000", 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001552793 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282917+0000", 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9567999999999997e-05 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282975+0000", 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7617000000000001e-05 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.313647+0000", 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030671894000000002 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s6 44/38 ECSubWrite(tid=62, reqid=client.4477.0:308, at_version=44'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273497+0000", 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 47.634891013000001, 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.040229292, 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273497+0000", 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273497+0000", 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273498+0000", 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2799999999999996e-07 2024-09-15T01:33:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273514+0000", 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5942000000000001e-05 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273538+0000", 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4705000000000001e-05 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273543+0000", 2024-09-15T01:33:33.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9649999999999999e-06 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281338+0000", 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0077948050000000001 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281343+0000", 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6009999999999997e-06 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281391+0000", 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7768000000000001e-05 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282868+0000", 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001477587 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282917+0000", 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9168000000000001e-05 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282987+0000", 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9294000000000004e-05 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.313726+0000", 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030739829 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=100, last_complete=54'36, committed=1, applied=1))", 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 37.811979342000001, 2024-09-15T01:33:33.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.063605834999999999, 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.01e-07 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096412+0000", 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.836e-06 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.159953+0000", 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.063540907999999993 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.159956+0000", 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0489999999999999e-06 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.159987+0000", 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0725999999999997e-05 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.160015+0000", 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7915e-05 2024-09-15T01:33:33.985 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:485 3.9s0 3:98b2f2fd:::smithi17529310-14:head [setxattr _header (57) in=7b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 37.744015650999998, 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40475048200000002, 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 485 2024-09-15T01:33:33.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6199999999999998e-07 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164375+0000", 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0650000000000001e-06 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164380+0000", 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9810000000000003e-06 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164383+0000", 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.241e-06 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165443+0000", 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010601409999999999 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165554+0000", 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011091600000000001 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:33.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165888+0000", 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000334125 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165960+0000", 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1601999999999996e-05 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165971+0000", 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0903999999999999e-05 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166005+0000", 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3525999999999997e-05 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166156+0000", 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000151077 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569047+0000", 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.402891102 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569123+0000", 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6340000000000004e-05 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:486 3.9s0 3:98b2f2fd:::smithi17529310-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 37.743989599999999, 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40586654100000003, 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:33.992 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 486 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2800000000000003e-07 2024-09-15T01:33:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164402+0000", 2024-09-15T01:33:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3440000000000001e-06 2024-09-15T01:33:33.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164409+0000", 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6839999999999999e-06 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164417+0000", 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9140000000000008e-06 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166021+0000", 2024-09-15T01:33:33.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016044189999999999 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166112+0000", 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0402000000000006e-05 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569421+0000", 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40330946099999998 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:33.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569534+0000", 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011237499999999999 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.570265+0000", 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00073161400000000001 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=106, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 37.742190522999998, 2024-09-15T01:33:33.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.10267822, 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0399999999999996e-07 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166203+0000", 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2889999999999999e-06 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268821+0000", 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.102618771 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:33.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268827+0000", 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1140000000000002e-06 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268865+0000", 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7492e-05 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268876+0000", 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.095e-05 2024-09-15T01:33:34.000 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=106, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.166241+0000", 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 37.742146742000003, 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.075762497999999998, 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166241+0000", 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166241+0000", 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166242+0000", 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6400000000000003e-07 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166244+0000", 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3309999999999998e-06 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.241879+0000", 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.075634921999999993 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.241898+0000", 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9082000000000001e-05 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.241991+0000", 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2833999999999999e-05 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.242004+0000", 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2865e-05 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=106, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:34.004 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.166248+0000", 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 37.742140190000001, 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40313786299999999, 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166248+0000", 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166248+0000", 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:34.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166249+0000", 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6700000000000003e-07 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166251+0000", 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.934e-06 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.568876+0000", 2024-09-15T01:33:34.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.402625446 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.568887+0000", 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0553e-05 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.568964+0000", 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7303000000000002e-05 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569386+0000", 2024-09-15T01:33:34.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00042205999999999998 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 81/31 ECSubWriteReply(tid=148, last_complete=81'55, committed=1, applied=1))", 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:23.088718+0000", 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 10.819670133000001, 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.42277780999999998, 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.008 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.088718+0000", 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.088718+0000", 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.088719+0000", 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1999999999999998e-07 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.088725+0000", 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7209999999999999e-06 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.511335+0000", 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.42261066600000002 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.511351+0000", 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5153e-05 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.511467+0000", 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011659600000000001 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.511496+0000", 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8753999999999999e-05 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:34.011 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:34.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:33:34.013 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:34.011+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:34.086 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-15T01:33:34.086 INFO:tasks.rados.rados.0.smithi175.stdout:352: done (6 left) 2024-09-15T01:33:34.087 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 50 current snap is 31 2024-09-15T01:33:34.087 INFO:tasks.rados.rados.0.smithi175.stdout:346: finishing write tid 2 to smithi17529310-38 2024-09-15T01:33:34.087 INFO:tasks.rados.rados.0.smithi175.stdout:353: seq_num 149 ranges {0=98304} 2024-09-15T01:33:34.087 INFO:tasks.rados.rados.0.smithi175.stdout:353: writing smithi17529310-50 from 0 to 98304 tid 1 2024-09-15T01:33:34.088 INFO:tasks.rados.rados.0.smithi175.stdout:354: read oid 29 snap -1 2024-09-15T01:33:34.088 INFO:tasks.rados.rados.0.smithi175.stdout:354: expect (ObjNum 123 snap 20 seq_num 123) 2024-09-15T01:33:34.088 INFO:tasks.rados.rados.0.smithi175.stdout:355: snap_create 2024-09-15T01:33:34.088 INFO:tasks.rados.rados.0.smithi175.stdout:346: finishing write tid 3 to smithi17529310-38 2024-09-15T01:33:34.088 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 38 v 211 (ObjNum 147 snap 31 seq_num 147) dirty exists 2024-09-15T01:33:34.088 INFO:tasks.rados.rados.0.smithi175.stdout:346: left oid 38 (ObjNum 147 snap 31 seq_num 147) 2024-09-15T01:33:34.088 INFO:tasks.rados.rados.0.smithi175.stdout:345: expect (ObjNum 115 snap 18 seq_num 115) 2024-09-15T01:33:34.114 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:34.112+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:34.131 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:34.131 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:34.131 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:34.131 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:34.140 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:33:34.157 INFO:tasks.rados.rados.0.smithi175.stdout:349: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-15T01:33:34.175 INFO:tasks.rados.rados.0.smithi175.stdout:347: expect (ObjNum 126 snap 20 seq_num 126) 2024-09-15T01:33:34.215 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:34.213+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:34.254 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:34.254 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:34.255 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:34.255 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:34.255 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:34.263 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:33:34.315 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:34.314+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:34.328 INFO:tasks.rados.rados.0.smithi175.stdout:348: expect (ObjNum 105 snap 10 seq_num 105) 2024-09-15T01:33:34.404 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:34.404 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:192 3.as0 3:5e9da7d3:::smithi17529310-1:head [copy-from ver 6] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.209181461999997, 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01403071, 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:34.405 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 192 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192423+0000", 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3140000000000002e-06 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192431+0000", 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2320000000000001e-06 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192444+0000", 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3239e-05 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192454+0000", 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2439999999999999e-06 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192781+0000", 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032732699999999998 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192904+0000", 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012317900000000001 2024-09-15T01:33:34.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.201873+0000", 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089691839999999998 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202377+0000", 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00050339000000000004 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202392+0000", 2024-09-15T01:33:34.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5201999999999999e-05 2024-09-15T01:33:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202869+0000", 2024-09-15T01:33:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047669800000000001 2024-09-15T01:33:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202888+0000", 2024-09-15T01:33:34.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9202e-05 2024-09-15T01:33:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203431+0000", 2024-09-15T01:33:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00054287499999999995 2024-09-15T01:33:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.206451+0000", 2024-09-15T01:33:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003020624 2024-09-15T01:33:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.411 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 48.154627251000001, 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013669871, 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:34.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246976+0000", 2024-09-15T01:33:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9800000000000002e-07 2024-09-15T01:33:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247000+0000", 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4314000000000001e-05 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247045+0000", 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4938000000000002e-05 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247053+0000", 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247132+0000", 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8799000000000002e-05 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247142+0000", 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0511e-05 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247515+0000", 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037233299999999997 2024-09-15T01:33:34.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247546+0000", 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1102000000000001e-05 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247585+0000", 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9152000000000001e-05 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247603+0000", 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-15T01:33:34.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.260645+0000", 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013041662000000001 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=55, reqid=client.4477.0:517, at_version=55'45, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 37.326069744000002, 2024-09-15T01:33:34.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016315308000000001, 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075533+0000", 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9199999999999999e-07 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075557+0000", 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3264e-05 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076900+0000", 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013436749999999999 2024-09-15T01:33:34.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076916+0000", 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5302999999999999e-05 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076970+0000", 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4265999999999998e-05 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076980+0000", 2024-09-15T01:33:34.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0521e-05 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077104+0000", 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012322599999999999 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077118+0000", 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4742e-05 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077164+0000", 2024-09-15T01:33:34.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5972e-05 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077228+0000", 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3291000000000001e-05 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.091848+0000", 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014620055999999999 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=56, reqid=client.4477.0:518, at_version=55'46, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 37.324748108999998, 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015201957, 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076855+0000", 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.057e-06 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076876+0000", 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0874e-05 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079870+0000", 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0029943259999999998 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079882+0000", 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2026e-05 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079958+0000", 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5263000000000001e-05 2024-09-15T01:33:34.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079969+0000", 2024-09-15T01:33:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.145e-05 2024-09-15T01:33:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.080116+0000", 2024-09-15T01:33:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014736600000000001 2024-09-15T01:33:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.080133+0000", 2024-09-15T01:33:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6776e-05 2024-09-15T01:33:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082691+0000", 2024-09-15T01:33:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025575229999999999 2024-09-15T01:33:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082740+0000", 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9373000000000003e-05 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.092056+0000", 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093159230000000003 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=57, reqid=client.4477.0:518, at_version=0'0, trim_to=0'0, roll_forward_to=55'46))", 2024-09-15T01:33:34.428 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 37.317824381999998, 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.77065714699999999, 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083779+0000", 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083792+0000", 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3478e-05 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853926+0000", 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.77013421000000004 2024-09-15T01:33:34.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853932+0000", 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2820000000000004e-06 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853992+0000", 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9886999999999998e-05 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854001+0000", 2024-09-15T01:33:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3700000000000001e-06 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854114+0000", 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000112651 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854152+0000", 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8518e-05 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854203+0000", 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1288000000000001e-05 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854223+0000", 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9884000000000001e-05 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854435+0000", 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021157399999999999 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.433 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s3 66/31 ECSubWrite(tid=130, reqid=osd.14.0:130, at_version=66'56, trim_to=0'0, roll_forward_to=60'52))", 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.320807384999998, 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016749074999999999, 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8199999999999999e-07 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080811+0000", 2024-09-15T01:33:34.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6435000000000001e-05 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082265+0000", 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001453415 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082270+0000", 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9470000000000002e-06 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082610+0000", 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00034021700000000003 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082618+0000", 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3969999999999994e-06 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084267+0000", 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001649069 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084292+0000", 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4125999999999999e-05 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084507+0000", 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0002154 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084539+0000", 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2208000000000001e-05 2024-09-15T01:33:34.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097544+0000", 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013004679 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 66/54 ECSubWrite(tid=240, reqid=osd.13.0:240, at_version=66'36, trim_to=0'0, roll_forward_to=56'33))", 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.320736668999999, 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020188490999999999, 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.439 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4600000000000001e-07 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080879+0000", 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3557000000000001e-05 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082859+0000", 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019801799999999998 2024-09-15T01:33:34.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082865+0000", 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8309999999999997e-06 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083375+0000", 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051010200000000002 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083383+0000", 2024-09-15T01:33:34.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7740000000000004e-06 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085762+0000", 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023785339999999999 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086258+0000", 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00049678099999999996 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086290+0000", 2024-09-15T01:33:34.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1612999999999997e-05 2024-09-15T01:33:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086368+0000", 2024-09-15T01:33:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8417000000000005e-05 2024-09-15T01:33:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101054+0000", 2024-09-15T01:33:34.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014685556000000001 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=62, reqid=osd.7.0:62, at_version=66'52, trim_to=0'0, roll_forward_to=58'50))", 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081340+0000", 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.320261851000001, 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016057799000000001, 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081340+0000", 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081340+0000", 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081341+0000", 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3900000000000004e-07 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081364+0000", 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3078000000000002e-05 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082876+0000", 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001512298 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082880+0000", 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1150000000000004e-06 2024-09-15T01:33:34.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082998+0000", 2024-09-15T01:33:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117386 2024-09-15T01:33:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083007+0000", 2024-09-15T01:33:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4849999999999998e-06 2024-09-15T01:33:34.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084480+0000", 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014727799999999999 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084509+0000", 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8878e-05 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084575+0000", 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5848999999999999e-05 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085049+0000", 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047466899999999999 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097398+0000", 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012348622 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=63, reqid=osd.7.0:63, at_version=66'54, trim_to=0'0, roll_forward_to=58'50))", 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.320234613, 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017023607, 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081386+0000", 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952e-05 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083054+0000", 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001668295 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083059+0000", 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9359999999999998e-06 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084544+0000", 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001484983 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084582+0000", 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7764e-05 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084807+0000", 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022473099999999999 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084975+0000", 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000167997 2024-09-15T01:33:34.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086254+0000", 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001279608 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086318+0000", 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3590000000000006e-05 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.098391+0000", 2024-09-15T01:33:34.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012073421000000001 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 66/31 ECSubWrite(tid=241, reqid=osd.13.0:241, at_version=66'51, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081392+0000", 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.320209784999999, 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017955004, 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081392+0000", 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081392+0000", 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081393+0000", 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7399999999999999e-07 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081416+0000", 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3807000000000001e-05 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083070+0000", 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001653416 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083073+0000", 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.58e-06 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083228+0000", 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000154671 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083236+0000", 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1960000000000008e-06 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085387+0000", 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021502230000000002 2024-09-15T01:33:34.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086257+0000", 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00087064300000000005 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086289+0000", 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2048000000000002e-05 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086344+0000", 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5031000000000002e-05 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.099347+0000", 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013003115000000001 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 66/31 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=66'53, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.083317+0000", 2024-09-15T01:33:34.461 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.318285100000001, 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017650290999999999, 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083317+0000", 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083317+0000", 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083318+0000", 2024-09-15T01:33:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9100000000000004e-07 2024-09-15T01:33:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083337+0000", 2024-09-15T01:33:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9488e-05 2024-09-15T01:33:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083371+0000", 2024-09-15T01:33:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3522000000000002e-05 2024-09-15T01:33:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083382+0000", 2024-09-15T01:33:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1715999999999999e-05 2024-09-15T01:33:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085431+0000", 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020484309999999999 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085439+0000", 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3389999999999993e-06 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085612+0000", 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017278700000000001 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086258+0000", 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00064593500000000004 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086290+0000", 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1813999999999997e-05 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086355+0000", 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5405000000000001e-05 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.100967+0000", 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014612263 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=64, reqid=client.4477.0:609, at_version=66'56, trim_to=0'0, roll_forward_to=66'52))", 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.087506+0000", 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.314096178, 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014356819999999999, 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087506+0000", 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087506+0000", 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087507+0000", 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.144e-06 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087633+0000", 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012627100000000001 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087682+0000", 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8551000000000003e-05 2024-09-15T01:33:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087690+0000", 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5979999999999999e-06 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087747+0000", 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7376000000000001e-05 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087758+0000", 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1065e-05 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.088106+0000", 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000347766 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.088114+0000", 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3850000000000002e-06 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091745+0000", 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003630968 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091795+0000", 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9922000000000001e-05 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101863+0000", 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010067774 2024-09-15T01:33:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 66/31 ECSubWrite(tid=244, reqid=osd.13.0:242, at_version=0'0, trim_to=0'0, roll_forward_to=66'53))", 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.089639+0000", 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.311962731000001, 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012039856, 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089639+0000", 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089639+0000", 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089640+0000", 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089649+0000", 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0459999999999994e-06 2024-09-15T01:33:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089681+0000", 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1588999999999999e-05 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089687+0000", 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9819999999999996e-06 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089743+0000", 2024-09-15T01:33:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5841999999999998e-05 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089751+0000", 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3790000000000006e-06 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089878+0000", 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012716299999999999 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091756+0000", 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018777080000000001 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091830+0000", 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4288000000000004e-05 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091848+0000", 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7448e-05 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101679+0000", 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0098317509999999997 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 66/54 ECSubWrite(tid=243, reqid=osd.13.0:240, at_version=0'0, trim_to=0'0, roll_forward_to=66'36))", 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.311950302, 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01224519, 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.48e-07 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089655+0000", 2024-09-15T01:33:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:33:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089726+0000", 2024-09-15T01:33:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0436000000000001e-05 2024-09-15T01:33:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089731+0000", 2024-09-15T01:33:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.164e-06 2024-09-15T01:33:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089770+0000", 2024-09-15T01:33:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9110999999999999e-05 2024-09-15T01:33:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089777+0000", 2024-09-15T01:33:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2599999999999999e-06 2024-09-15T01:33:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.090039+0000", 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000261506 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091756+0000", 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001717704 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091831+0000", 2024-09-15T01:33:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4177000000000005e-05 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091855+0000", 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4300000000000001e-05 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101897+0000", 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010042163 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.as0 76/69 ECSubWriteReply(tid=60, last_complete=76'39, committed=1, applied=1))", 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:18.085798+0000", 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 16.315804202999999, 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19697366399999999, 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085798+0000", 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085798+0000", 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085799+0000", 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9199999999999999e-07 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085804+0000", 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0440000000000003e-06 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282638+0000", 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19683383099999999 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282645+0000", 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9800000000000001e-06 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282743+0000", 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8517999999999994e-05 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282772+0000", 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8399e-05 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/54 ECSubWrite(tid=303, reqid=client.4477.0:846, at_version=86'54, trim_to=0'0, roll_forward_to=85'52))", 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.133620+0000", 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.2679824210000001, 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010527086, 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133620+0000", 2024-09-15T01:33:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133620+0000", 2024-09-15T01:33:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133620+0000", 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.43e-07 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133641+0000", 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1316000000000001e-05 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133695+0000", 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3981999999999999e-05 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133702+0000", 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.849e-06 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133755+0000", 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3094999999999999e-05 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133768+0000", 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2534000000000001e-05 2024-09-15T01:33:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134038+0000", 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000270571 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134053+0000", 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5091e-05 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134099+0000", 2024-09-15T01:33:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5439999999999999e-05 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134122+0000", 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3150999999999999e-05 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.144147+0000", 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010024913999999999 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/54 ECSubWrite(tid=304, reqid=client.4477.0:847, at_version=86'55, trim_to=0'0, roll_forward_to=85'52))", 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.134900+0000", 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.2667023669999997, 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018173871000000001, 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134900+0000", 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134900+0000", 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134901+0000", 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0359999999999999e-06 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134932+0000", 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0868999999999999e-05 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134994+0000", 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2161000000000003e-05 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135000+0000", 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2319999999999998e-06 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135199+0000", 2024-09-15T01:33:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000198438 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135207+0000", 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0830000000000004e-06 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135318+0000", 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110881 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135331+0000", 2024-09-15T01:33:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3409e-05 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135370+0000", 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9144000000000003e-05 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135466+0000", 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5527999999999997e-05 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.153074+0000", 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01760809 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/54 ECSubWrite(tid=305, reqid=client.4477.0:848, at_version=86'56, trim_to=0'0, roll_forward_to=85'52))", 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.135237+0000", 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.2663647620000003, 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018127175999999998, 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135237+0000", 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135237+0000", 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135238+0000", 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3600000000000004e-07 2024-09-15T01:33:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135253+0000", 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5339e-05 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135283+0000", 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9445e-05 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135290+0000", 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8329999999999996e-06 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135356+0000", 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5982000000000006e-05 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135363+0000", 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7030000000000006e-06 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135446+0000", 2024-09-15T01:33:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2695999999999996e-05 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135454+0000", 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1669999999999999e-06 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135511+0000", 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7265000000000001e-05 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135519+0000", 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6739999999999997e-06 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.153365+0000", 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017845535999999999 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/54 ECSubWrite(tid=306, reqid=client.4477.0:848, at_version=0'0, trim_to=0'0, roll_forward_to=86'56))", 2024-09-15T01:33:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.136794+0000", 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.2648079970000001, 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016821658999999999, 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136794+0000", 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136794+0000", 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136795+0000", 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0890000000000001e-06 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136806+0000", 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0301e-05 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136844+0000", 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8002e-05 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136852+0000", 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3979999999999999e-06 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136958+0000", 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010578600000000001 2024-09-15T01:33:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136969+0000", 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0767999999999999e-05 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137111+0000", 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142922 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137120+0000", 2024-09-15T01:33:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137317+0000", 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019695499999999999 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137331+0000", 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.385e-05 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.153616+0000", 2024-09-15T01:33:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016284666 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 86/31 ECSubWrite(tid=104, reqid=client.4477.0:836, at_version=0'0, trim_to=0'0, roll_forward_to=86'87))", 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.137961+0000", 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.2636409730000002, 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015483966, 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137961+0000", 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137961+0000", 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137962+0000", 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4200000000000004e-07 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137975+0000", 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2993000000000001e-05 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138014+0000", 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9050000000000001e-05 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138021+0000", 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1130000000000003e-06 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138076+0000", 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4867999999999997e-05 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138087+0000", 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0623e-05 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138238+0000", 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000150944 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.139771+0000", 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00153374 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.139849+0000", 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7737999999999993e-05 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.139880+0000", 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0809999999999998e-05 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.153445+0000", 2024-09-15T01:33:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013565145000000001 2024-09-15T01:33:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.514 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:34.516 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:33:34.516 INFO:tasks.rados.rados.0.smithi175.stdout:350: expect (ObjNum 103 snap 10 seq_num 103) 2024-09-15T01:33:34.516 INFO:tasks.rados.rados.0.smithi175.stdout:353: finishing write tid 1 to smithi17529310-50 2024-09-15T01:33:34.517 INFO:tasks.rados.rados.0.smithi175.stdout:353: finishing write tid 2 to smithi17529310-50 2024-09-15T01:33:34.517 INFO:tasks.rados.rados.0.smithi175.stdout:353: finishing write tid 3 to smithi17529310-50 2024-09-15T01:33:34.517 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 50 v 45 (ObjNum 149 snap 31 seq_num 149) dirty exists 2024-09-15T01:33:34.517 INFO:tasks.rados.rados.0.smithi175.stdout:353: left oid 50 (ObjNum 149 snap 31 seq_num 149) 2024-09-15T01:33:34.517 INFO:tasks.rados.rados.0.smithi175.stdout:354: expect (ObjNum 123 snap 20 seq_num 123) 2024-09-15T01:33:34.517 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:34.514+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:34.617 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:34.615+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:34.653 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:34.653 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:34.654 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:34.654 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:34.662 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:33:34.717 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:34.715+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:34.759 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:34.759 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:34.760 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:34.760 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:34.760 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:34.767 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:33:34.818 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:34.817+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:34.906 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:34.906 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:108 3.ds0 3:b063f44c:::smithi17529310-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 58.657377928999999, 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013756292, 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:34.907 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 108 2024-09-15T01:33:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246459+0000", 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6800000000000001e-07 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246461+0000", 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.131e-06 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246468+0000", 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5529999999999998e-06 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246475+0000", 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5830000000000001e-06 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.248249+0000", 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017739990000000001 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.248333+0000", 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4288000000000003e-05 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.249154+0000", 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00082014900000000005 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.249218+0000", 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4519999999999999e-05 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.260215+0000", 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010996601 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 48.662613426, 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014051065, 2024-09-15T01:33:34.912 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241243+0000", 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0126e-05 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241422+0000", 2024-09-15T01:33:34.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017906899999999999 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241429+0000", 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4139999999999999e-06 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241500+0000", 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1098000000000003e-05 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241510+0000", 2024-09-15T01:33:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8470000000000008e-06 2024-09-15T01:33:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241787+0000", 2024-09-15T01:33:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027729700000000003 2024-09-15T01:33:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242007+0000", 2024-09-15T01:33:34.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000220134 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242146+0000", 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013846799999999999 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242278+0000", 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0001326 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.255274+0000", 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01299575 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 48.660155080999999, 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014188612, 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243682+0000", 2024-09-15T01:33:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0299999999999999e-07 2024-09-15T01:33:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243699+0000", 2024-09-15T01:33:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7575999999999999e-05 2024-09-15T01:33:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244099+0000", 2024-09-15T01:33:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039909500000000003 2024-09-15T01:33:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244108+0000", 2024-09-15T01:33:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3209999999999994e-06 2024-09-15T01:33:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244168+0000", 2024-09-15T01:33:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9688999999999997e-05 2024-09-15T01:33:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244180+0000", 2024-09-15T01:33:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2031000000000001e-05 2024-09-15T01:33:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245504+0000", 2024-09-15T01:33:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013241520000000001 2024-09-15T01:33:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245537+0000", 2024-09-15T01:33:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3287000000000002e-05 2024-09-15T01:33:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245645+0000", 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108435 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245664+0000", 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8329999999999999e-05 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.257870+0000", 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012206092999999999 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.924 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 48.656871498000001, 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013797696999999999, 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:34.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246966+0000", 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000002e-07 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246979+0000", 2024-09-15T01:33:34.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3111e-05 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247017+0000", 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8096999999999998e-05 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247025+0000", 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9100000000000005e-06 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247095+0000", 2024-09-15T01:33:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0049000000000006e-05 2024-09-15T01:33:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247104+0000", 2024-09-15T01:33:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7919999999999998e-06 2024-09-15T01:33:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247217+0000", 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011299 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247376+0000", 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015968800000000001 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247417+0000", 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0228999999999997e-05 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248151+0000", 2024-09-15T01:33:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00073475999999999999 2024-09-15T01:33:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.260763+0000", 2024-09-15T01:33:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012611282999999999 2024-09-15T01:33:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:283 3.ds0 3:bfa066a3:::smithi17529310-39:head [copy-from ver 2] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 48.645070760000003, 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015911307999999999, 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 283 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258768+0000", 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0980000000000001e-06 2024-09-15T01:33:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258779+0000", 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1272e-05 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258795+0000", 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5679999999999999e-05 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258802+0000", 2024-09-15T01:33:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9029999999999998e-06 2024-09-15T01:33:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258879+0000", 2024-09-15T01:33:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7035999999999999e-05 2024-09-15T01:33:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.259139+0000", 2024-09-15T01:33:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260456 2024-09-15T01:33:34.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268247+0000", 2024-09-15T01:33:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0091076109999999998 2024-09-15T01:33:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268518+0000", 2024-09-15T01:33:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027146499999999998 2024-09-15T01:33:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268542+0000", 2024-09-15T01:33:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3601e-05 2024-09-15T01:33:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268616+0000", 2024-09-15T01:33:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3944999999999998e-05 2024-09-15T01:33:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.269087+0000", 2024-09-15T01:33:34.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047095399999999999 2024-09-15T01:33:34.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.274577+0000", 2024-09-15T01:33:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0054903019999999999 2024-09-15T01:33:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.274677+0000", 2024-09-15T01:33:34.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9984999999999999e-05 2024-09-15T01:33:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 44/31 ECSubWriteReply(tid=43, last_complete=44'32, committed=1, applied=1))", 2024-09-15T01:33:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 48.628389894000001, 2024-09-15T01:33:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.192594981, 2024-09-15T01:33:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.941 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275447+0000", 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2099999999999995e-07 2024-09-15T01:33:34.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275452+0000", 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2050000000000004e-06 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.467889+0000", 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19243722999999999 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.467922+0000", 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3237e-05 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.468017+0000", 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4543999999999995e-05 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.468041+0000", 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4844000000000001e-05 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.944 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 66/31 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=66'53, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.821154668999998, 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014557652000000001, 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:34.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082683+0000", 2024-09-15T01:33:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2999999999999999e-07 2024-09-15T01:33:34.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082704+0000", 2024-09-15T01:33:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1381000000000001e-05 2024-09-15T01:33:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082736+0000", 2024-09-15T01:33:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2239000000000001e-05 2024-09-15T01:33:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082743+0000", 2024-09-15T01:33:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6449999999999999e-06 2024-09-15T01:33:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082927+0000", 2024-09-15T01:33:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018380800000000001 2024-09-15T01:33:34.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082934+0000", 2024-09-15T01:33:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1600000000000001e-06 2024-09-15T01:33:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083133+0000", 2024-09-15T01:33:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019963000000000001 2024-09-15T01:33:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083536+0000", 2024-09-15T01:33:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040219000000000002 2024-09-15T01:33:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083587+0000", 2024-09-15T01:33:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1600000000000001e-05 2024-09-15T01:33:34.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083644+0000", 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6753999999999999e-05 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097239+0000", 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013595415 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:609 3.ds0 3:b50d79c0:::smithi17529310-18:head [append 0~770048] snapc 1f=[1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.085186+0000", 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.818649919999999, 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013710833, 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 609 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085186+0000", 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085186+0000", 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085188+0000", 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.694e-06 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085539+0000", 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035108399999999999 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085555+0000", 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6127999999999999e-05 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085565+0000", 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6819999999999998e-06 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085613+0000", 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7478999999999997e-05 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085720+0000", 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010775799999999999 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087279+0000", 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001558729 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.092083+0000", 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.004803501 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.092092+0000", 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3389999999999999e-06 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.094255+0000", 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021635970000000002 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.096016+0000", 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017606130000000001 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.096085+0000", 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9275000000000003e-05 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.098897+0000", 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028119540000000002 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.957 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s5 71/69 ECSubWrite(tid=73, reqid=client.4477.0:701, at_version=71'45, trim_to=0'0, roll_forward_to=70'43))", 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.321241+0000", 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.582594932999999, 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014395221999999999, 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321241+0000", 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321241+0000", 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321242+0000", 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.36e-07 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327486+0000", 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0062439130000000002 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327562+0000", 2024-09-15T01:33:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6878999999999994e-05 2024-09-15T01:33:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327570+0000", 2024-09-15T01:33:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3520000000000001e-06 2024-09-15T01:33:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327617+0000", 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6839999999999999e-05 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327627+0000", 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.927e-06 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328144+0000", 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051762900000000003 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328237+0000", 2024-09-15T01:33:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3153999999999996e-05 2024-09-15T01:33:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328549+0000", 2024-09-15T01:33:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000311873 2024-09-15T01:33:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328572+0000", 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2751999999999998e-05 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.335637+0000", 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0070647669999999996 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:823 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [copy-from ver 45] snapc 30=[30,2e,2c,2b,29,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-15T01:33:34.963 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:27.126788+0000", 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 7.7770486309999995, 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.02620683, 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 823 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126788+0000", 2024-09-15T01:33:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126788+0000", 2024-09-15T01:33:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126790+0000", 2024-09-15T01:33:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9360000000000002e-06 2024-09-15T01:33:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126797+0000", 2024-09-15T01:33:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0330000000000003e-06 2024-09-15T01:33:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126824+0000", 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7637e-05 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126834+0000", 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.268e-06 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126932+0000", 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7882999999999998e-05 2024-09-15T01:33:34.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127109+0000", 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017735299999999999 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.142551+0000", 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015441939 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143350+0000", 2024-09-15T01:33:34.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00079953299999999995 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143360+0000", 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0098999999999999e-05 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143663+0000", 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030244499999999998 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143835+0000", 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017216200000000001 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.152928+0000", 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0090929219999999998 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.152995+0000", 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6619999999999996e-05 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.970 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:824 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 7.7769905770000003, 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016980479, 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 824 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.971 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6199999999999998e-07 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126849+0000", 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6800000000000002e-06 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127309+0000", 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045968300000000001 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127316+0000", 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9500000000000004e-06 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127358+0000", 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1990999999999998e-05 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127413+0000", 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5448999999999997e-05 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143574+0000", 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016161045999999998 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143678+0000", 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103719 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143826+0000", 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014839900000000001 2024-09-15T01:33:34.975 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:834 3.ds0 3:bfa066a3:::smithi17529310-39:head [append 0~475136] snapc 32=[32,30,2e,2c,2b,29,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.122940+0000", 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.7808964439999997, 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01465559, 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 834 2024-09-15T01:33:34.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.122940+0000", 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.122940+0000", 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.122941+0000", 2024-09-15T01:33:34.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.187e-06 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123093+0000", 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000152087 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123103+0000", 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4339999999999998e-06 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123108+0000", 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123144+0000", 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5818000000000002e-05 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123231+0000", 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7831000000000007e-05 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123712+0000", 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00048043899999999999 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123895+0000", 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018332600000000001 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123903+0000", 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8849999999999999e-06 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.124026+0000", 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012307799999999999 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.124343+0000", 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031695200000000001 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137541+0000", 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013197690999999999 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137596+0000", 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4832e-05 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:837 3.ds0 3:bfa066a3:::smithi17529310-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.125675+0000", 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.7781617269999996, 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016649648999999999, 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:34.982 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 837 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125675+0000", 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125675+0000", 2024-09-15T01:33:34.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125675+0000", 2024-09-15T01:33:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6499999999999999e-07 2024-09-15T01:33:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125678+0000", 2024-09-15T01:33:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0800000000000002e-06 2024-09-15T01:33:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125688+0000", 2024-09-15T01:33:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9580000000000002e-06 2024-09-15T01:33:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125694+0000", 2024-09-15T01:33:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4670000000000003e-06 2024-09-15T01:33:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.127414+0000", 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017203640000000001 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.127494+0000", 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0179999999999995e-05 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138114+0000", 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010619161 2024-09-15T01:33:34.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138176+0000", 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2644999999999994e-05 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.142324+0000", 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0041481290000000004 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:842 3.ds0 3:b063f44c:::smithi17529310-36:head [read 0~2949120,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.125725+0000", 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.7781117809999998, 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013984874, 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 842 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125725+0000", 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125725+0000", 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125725+0000", 2024-09-15T01:33:34.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1800000000000001e-07 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125728+0000", 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125734+0000", 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6789999999999999e-06 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125738+0000", 2024-09-15T01:33:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.126e-06 2024-09-15T01:33:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.128322+0000", 2024-09-15T01:33:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025834970000000001 2024-09-15T01:33:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.128409+0000", 2024-09-15T01:33:34.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7362999999999997e-05 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.139709+0000", 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01130014 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:843 3.ds0 3:b063f44c:::smithi17529310-36:head [read 0~2949120,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.125742+0000", 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.7780942749999999, 2024-09-15T01:33:34.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015933147000000002, 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 843 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125742+0000", 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125742+0000", 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125742+0000", 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9900000000000002e-07 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125745+0000", 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2900000000000001e-06 2024-09-15T01:33:34.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125751+0000", 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0310000000000004e-06 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125755+0000", 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8190000000000002e-06 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.129247+0000", 2024-09-15T01:33:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0034922130000000001 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.129335+0000", 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8098999999999998e-05 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.141675+0000", 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012340396 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:34.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 90/31 ECSubWriteReply(tid=117, last_complete=90'103, committed=1, applied=1))", 2024-09-15T01:33:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:32.087896+0000", 2024-09-15T01:33:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.8159407359999999, 2024-09-15T01:33:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.78023241700000001, 2024-09-15T01:33:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:34.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.087896+0000", 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.087896+0000", 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.087896+0000", 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8899999999999999e-07 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.087900+0000", 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0180000000000003e-06 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.867952+0000", 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.78005183600000005 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.867963+0000", 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1214e-05 2024-09-15T01:33:34.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.868082+0000", 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118872 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.868128+0000", 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5887999999999998e-05 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:888 3.ds0 3:b50d79c0:::smithi17529310-18:head [read 0~2228224,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:33:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:33.082507+0000", 2024-09-15T01:33:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.821329612, 2024-09-15T01:33:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024728772999999999, 2024-09-15T01:33:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.001 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 888 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082507+0000", 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082507+0000", 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082507+0000", 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.46e-07 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082509+0000", 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1339999999999998e-06 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082514+0000", 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7779999999999998e-06 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082518+0000", 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2289999999999997e-06 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082626+0000", 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107278 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082630+0000", 2024-09-15T01:33:35.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3000000000000003e-06 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.085853+0000", 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0032233330000000001 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.085981+0000", 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127572 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.107236+0000", 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021254802999999999 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:898 3.ds0 3:b063f44c:::smithi17529310-36:19 [read 0~589824,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:33.083177+0000", 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.8206593469999999, 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024976851000000001, 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:35.006 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 898 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083177+0000", 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083177+0000", 2024-09-15T01:33:35.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083178+0000", 2024-09-15T01:33:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.006e-06 2024-09-15T01:33:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083184+0000", 2024-09-15T01:33:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9150000000000001e-06 2024-09-15T01:33:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083199+0000", 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4789e-05 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083205+0000", 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2330000000000003e-06 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083312+0000", 2024-09-15T01:33:35.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010702899999999999 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083319+0000", 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0319999999999998e-06 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.086360+0000", 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030408840000000002 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.086443+0000", 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2886999999999994e-05 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.108154+0000", 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021711075999999999 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:899 3.ds0 3:b063f44c:::smithi17529310-36:19 [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:33.083217+0000", 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.8206190740000001, 2024-09-15T01:33:35.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.025600806, 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 899 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083217+0000", 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083217+0000", 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083218+0000", 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5199999999999997e-07 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083221+0000", 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9249999999999999e-06 2024-09-15T01:33:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083228+0000", 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5340000000000002e-06 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083234+0000", 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1970000000000004e-06 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083361+0000", 2024-09-15T01:33:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127136 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083366+0000", 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8949999999999997e-06 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.093115+0000", 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0097499279999999997 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.093210+0000", 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4549000000000006e-05 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.108818+0000", 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01560809 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:900 3.ds0 3:b063f44c:::smithi17529310-36:19 [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:33:35.016 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:33.083238+0000", 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.820598084, 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.026108421, 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 900 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083238+0000", 2024-09-15T01:33:35.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083238+0000", 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083239+0000", 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5200000000000002e-07 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083242+0000", 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0280000000000001e-06 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083248+0000", 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6900000000000003e-06 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083253+0000", 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1280000000000001e-06 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083395+0000", 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014244099999999999 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083403+0000", 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8169999999999999e-06 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.093539+0000", 2024-09-15T01:33:35.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010135933999999999 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.093696+0000", 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000157427 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.109347+0000", 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015650503999999999 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.021 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.022 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:35.023 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:33:35.024 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:35.021+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:35.088 INFO:tasks.rados.rados.0.smithi175.stdout:345: done (8 left) 2024-09-15T01:33:35.089 INFO:tasks.rados.rados.0.smithi175.stdout:346: done (7 left) 2024-09-15T01:33:35.089 INFO:tasks.rados.rados.0.smithi175.stdout:347: done (6 left) 2024-09-15T01:33:35.089 INFO:tasks.rados.rados.0.smithi175.stdout:348: done (5 left) 2024-09-15T01:33:35.089 INFO:tasks.rados.rados.0.smithi175.stdout:349: done (4 left) 2024-09-15T01:33:35.089 INFO:tasks.rados.rados.0.smithi175.stdout:350: done (3 left) 2024-09-15T01:33:35.089 INFO:tasks.rados.rados.0.smithi175.stdout:353: done (2 left) 2024-09-15T01:33:35.089 INFO:tasks.rados.rados.0.smithi175.stdout:354: done (1 left) 2024-09-15T01:33:35.089 INFO:tasks.rados.rados.0.smithi175.stdout:355: done (0 left) 2024-09-15T01:33:35.089 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 7 current snap is 32 2024-09-15T01:33:35.089 INFO:tasks.rados.rados.0.smithi175.stdout:356: seq_num 150 ranges {2703360=114688} 2024-09-15T01:33:35.105 INFO:tasks.rados.rados.0.smithi175.stdout:356: writing smithi17529310-7 from 2703360 to 2818048 tid 1 2024-09-15T01:33:35.105 INFO:tasks.rados.rados.0.smithi175.stdout:357: copy_from oid 21 from oid 30 current snap is 32 2024-09-15T01:33:35.105 INFO:tasks.rados.rados.0.smithi175.stdout:358: read oid 45 snap -1 2024-09-15T01:33:35.105 INFO:tasks.rados.rados.0.smithi175.stdout:358: expect (ObjNum 68 snap 3 seq_num 68) 2024-09-15T01:33:35.105 INFO:tasks.rados.rados.0.smithi175.stdout:359: delete oid 34 current snap is 32 2024-09-15T01:33:35.110 INFO:tasks.rados.rados.0.smithi175.stdout:356: finishing write tid 1 to smithi17529310-7 2024-09-15T01:33:35.110 INFO:tasks.rados.rados.0.smithi175.stdout:356: finishing write tid 2 to smithi17529310-7 2024-09-15T01:33:35.113 INFO:tasks.rados.rados.0.smithi175.stdout:357: finishing copy_from racing read to smithi17529310-21 2024-09-15T01:33:35.113 INFO:tasks.rados.rados.0.smithi175.stdout:359: done (3 left) 2024-09-15T01:33:35.114 INFO:tasks.rados.rados.0.smithi175.stdout:360: snap_create 2024-09-15T01:33:35.115 INFO:tasks.rados.rados.0.smithi175.stdout:357: finishing copy_from to smithi17529310-21 2024-09-15T01:33:35.115 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 21 v 60 (ObjNum 65 snap 0 seq_num 1516269409) dirty exists 2024-09-15T01:33:35.117 INFO:tasks.rados.rados.0.smithi175.stdout:358: expect (ObjNum 68 snap 3 seq_num 68) 2024-09-15T01:33:35.124 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:35.122+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:35.130 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:35.130 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:35.130 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:35.130 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:35.139 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:33:35.171 INFO:tasks.rados.rados.0.smithi175.stdout:356: finishing write tid 3 to smithi17529310-7 2024-09-15T01:33:35.171 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 7 v 80 (ObjNum 150 snap 32 seq_num 150) dirty exists 2024-09-15T01:33:35.171 INFO:tasks.rados.rados.0.smithi175.stdout:356: left oid 7 (ObjNum 150 snap 32 seq_num 150) 2024-09-15T01:33:35.176 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:33:35.224 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:35.223+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:35.236 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:35.236 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:35.236 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:35.236 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:35.236 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:35.244 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:33:35.316 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:35.316 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:35.328 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:35.327+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:35.336 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:35.337 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:35.357 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:35.357 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:35.377 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:99 3.fs0 3:f630fe6e:::smithi17529310-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 59.148044400000003, 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055995037999999997, 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99 2024-09-15T01:33:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235203+0000", 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4000000000000002e-07 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235205+0000", 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3e-06 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235212+0000", 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4389999999999997e-06 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235216+0000", 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.031e-06 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.236877+0000", 2024-09-15T01:33:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00166088 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.236933+0000", 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6873999999999998e-05 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255906+0000", 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018972066999999999 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255957+0000", 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1467e-05 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291198+0000", 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035240540000000001 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:114 3.fs0 3:f8072b57:::smithi17529310-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 59.129604479000001, 2024-09-15T01:33:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048699039999999999, 2024-09-15T01:33:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 114 2024-09-15T01:33:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253643+0000", 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6899999999999998e-07 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253646+0000", 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253652+0000", 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3369999999999996e-06 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253656+0000", 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.031e-06 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255410+0000", 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017543420000000001 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255505+0000", 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4704999999999995e-05 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290833+0000", 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035327711999999997 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290893+0000", 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0384999999999999e-05 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.302341+0000", 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011448274 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:145 3.cs0 3:34a45b5f:::smithi17529310-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 59.085377911000002, 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15351202899999999, 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 145 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0500000000000004e-07 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297871+0000", 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9769999999999999e-06 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297878+0000", 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9120000000000001e-06 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297882+0000", 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0910000000000003e-06 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.298932+0000", 2024-09-15T01:33:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00104904 2024-09-15T01:33:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.299009+0000", 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7095000000000006e-05 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.306936+0000", 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0079278730000000002 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.307022+0000", 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5254999999999996e-05 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.451381+0000", 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14435928100000001 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:312 3.cs0 3:37dffec7:::smithi17529310-13:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 49.105054295999999, 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770305000000002, 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 312 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0200000000000002e-07 2024-09-15T01:33:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278196+0000", 2024-09-15T01:33:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7319999999999998e-06 2024-09-15T01:33:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278216+0000", 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0375999999999999e-05 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278221+0000", 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4889999999999997e-06 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279668+0000", 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014473929999999999 2024-09-15T01:33:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279753+0000", 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4567999999999994e-05 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280213+0000", 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00046030399999999998 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280327+0000", 2024-09-15T01:33:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011404399999999999 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280342+0000", 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4403000000000001e-05 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280467+0000", 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000125594 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280562+0000", 2024-09-15T01:33:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5317999999999998e-05 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328870+0000", 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048307782 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328963+0000", 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2700000000000004e-05 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:313 3.cs0 3:37dffec7:::smithi17529310-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 49.105021415000003, 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053267040000000002, 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 313 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278226+0000", 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7100000000000002e-07 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278228+0000", 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1859999999999999e-06 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278235+0000", 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9020000000000002e-06 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278240+0000", 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9540000000000003e-06 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280404+0000", 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021641469999999999 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280503+0000", 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9090000000000002e-05 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328996+0000", 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048492533999999997 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.329053+0000", 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6997999999999999e-05 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.331493+0000", 2024-09-15T01:33:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024397580000000002 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=95, last_complete=44'34, committed=1, applied=1))", 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 49.101617072000003, 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047344538999999998, 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7700000000000001e-07 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281632+0000", 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.562e-06 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328775+0000", 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047143813 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328779+0000", 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.895e-06 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328803+0000", 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4076000000000001e-05 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328974+0000", 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017101599999999999 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:324 3.cs0 3:34a45b5f:::smithi17529310-48:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 49.052601965000001, 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18862163400000001, 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 324 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699999999999999e-07 2024-09-15T01:33:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330648+0000", 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.694e-06 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330654+0000", 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4090000000000002e-06 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330659+0000", 2024-09-15T01:33:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3210000000000004e-06 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332198+0000", 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015388699999999999 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332262+0000", 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4196000000000007e-05 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332785+0000", 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052292099999999998 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332870+0000", 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5049999999999994e-05 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332877+0000", 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7769999999999997e-06 2024-09-15T01:33:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332902+0000", 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5012e-05 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332962+0000", 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0649999999999997e-05 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519220+0000", 2024-09-15T01:33:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18625739199999999 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519267+0000", 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6875000000000001e-05 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:325 3.cs0 3:34a45b5f:::smithi17529310-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 49.052583902000002, 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.189890015, 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 325 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330667+0000", 2024-09-15T01:33:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4410000000000002e-06 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330673+0000", 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5429999999999999e-06 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330676+0000", 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.683e-06 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333439+0000", 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002763415 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333492+0000", 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2760000000000003e-05 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519496+0000", 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18600408199999999 2024-09-15T01:33:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519560+0000", 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3660999999999999e-05 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.520553+0000", 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00099301399999999996 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=100, last_complete=44'39, committed=1, applied=1))", 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 49.050181858999998, 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18640773799999999, 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5999999999999999e-08 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333066+0000", 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.043e-06 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519088+0000", 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18602205399999999 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519092+0000", 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0890000000000002e-06 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519171+0000", 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8857999999999996e-05 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519473+0000", 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030159799999999998 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 47/31 ECSubWriteReply(tid=113, last_complete=47'51, committed=1, applied=1))", 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.123360048000002, 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.63648575600000001, 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-07 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259890+0000", 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5859999999999999e-06 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.895924+0000", 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.63603432699999995 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.895929+0000", 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1340000000000003e-06 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.896000+0000", 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0687999999999998e-05 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.896373+0000", 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037265200000000001 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 48/31 ECSubWriteReply(tid=128, last_complete=48'57, committed=1, applied=1))", 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 45.076544558999998, 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18111660500000001, 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306703+0000", 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3e-07 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306707+0000", 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7500000000000001e-06 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487741+0000", 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18103418199999999 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487745+0000", 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2189999999999998e-06 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487789+0000", 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3430000000000003e-05 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487819+0000", 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0394000000000001e-05 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 53/31 ECSubWriteReply(tid=146, last_complete=53'74, committed=1, applied=1))", 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 40.182703109999999, 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.74248160299999999, 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-07 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200550+0000", 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5910000000000003e-06 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.942942+0000", 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.74239190499999996 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.942965+0000", 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2983999999999999e-05 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.943000+0000", 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5336999999999997e-05 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.943025+0000", 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5250999999999999e-05 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.7s6 55/52 [PushOp(3:e6e8875f:::smithi17529310-31:head, version: 50'31, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:e6e8875f:::smithi17529310-31:head@50'31, size: 1949696, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 38.284988505000001, 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18071695199999999, 2024-09-15T01:33:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098259+0000", 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0200000000000001e-07 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098545+0000", 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028574699999999999 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278693+0000", 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.180148009 2024-09-15T01:33:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278700+0000", 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.888e-06 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278803+0000", 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102766 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278975+0000", 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017284000000000001 2024-09-15T01:33:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:524 3.fs0 3:f8e751c8:::smithi17529310-17:head [setxattr _header (57) in=7b] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 37.289224032, 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.115556992, 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 524 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2200000000000004e-07 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094028+0000", 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9710000000000003e-06 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094040+0000", 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1066e-05 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094044+0000", 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8770000000000001e-06 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095122+0000", 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010771839999999999 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095188+0000", 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6298999999999997e-05 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095521+0000", 2024-09-15T01:33:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033339600000000002 2024-09-15T01:33:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095614+0000", 2024-09-15T01:33:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2295999999999999e-05 2024-09-15T01:33:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095620+0000", 2024-09-15T01:33:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8000000000000001e-06 2024-09-15T01:33:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095653+0000", 2024-09-15T01:33:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2867000000000002e-05 2024-09-15T01:33:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095740+0000", 2024-09-15T01:33:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6300999999999999e-05 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209497+0000", 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.113757184 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209580+0000", 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3128999999999996e-05 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:525 3.fs0 3:f8e751c8:::smithi17529310-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 37.289198206999998, 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11664384, 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 525 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6100000000000001e-07 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094051+0000", 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1950000000000002e-06 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094058+0000", 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4350000000000002e-06 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094063+0000", 2024-09-15T01:33:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2039999999999996e-06 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095664+0000", 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001600642 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095725+0000", 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1707999999999999e-05 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209923+0000", 2024-09-15T01:33:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.114197744 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209997+0000", 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3341000000000001e-05 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.210693+0000", 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00069601000000000005 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:691 3.fs0 3:f73c1a72:::smithi17529310-29:head [copy-from ver 48] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.296702+0000", 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.086544628999999, 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.042728760999999997, 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 691 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296702+0000", 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296702+0000", 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296703+0000", 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296704+0000", 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7039999999999999e-06 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296708+0000", 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0439999999999997e-06 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296710+0000", 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0200000000000001e-06 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.299482+0000", 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027719820000000001 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.299578+0000", 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5361999999999999e-05 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321672+0000", 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022094721000000001 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322236+0000", 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00056322499999999999 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322250+0000", 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.438e-05 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322630+0000", 2024-09-15T01:33:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00038026599999999998 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322764+0000", 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013368899999999999 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.339358+0000", 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016593988 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.339431+0000", 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3041999999999996e-05 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:746 3.9s0 3:98b2f2fd:::smithi17529310-14:head [append 0~0] snapc 27=[27,26,24,22,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:18.078177+0000", 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.305069752000001, 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0091383140000001, 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 746 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078177+0000", 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078177+0000", 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078178+0000", 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1799999999999995e-07 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078181+0000", 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.038e-06 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078187+0000", 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4760000000000003e-06 2024-09-15T01:33:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078191+0000", 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7189999999999999e-06 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078448+0000", 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025739300000000002 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op must wait for map", 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078457+0000", 2024-09-15T01:33:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7590000000000003e-06 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.082114+0000", 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036565579999999999 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.082117+0000", 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0299999999999998e-06 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.084154+0000", 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0020370750000001 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.084312+0000", 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015769300000000001 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.084835+0000", 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052329899999999996 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.085132+0000", 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029761499999999999 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.085143+0000", 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0951000000000001e-05 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.085193+0000", 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9369000000000001e-05 2024-09-15T01:33:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.086029+0000", 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00083588200000000003 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.086781+0000", 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00075188200000000005 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087315+0000", 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00053495699999999997 2024-09-15T01:33:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:747 3.9s0 3:98b2f2fd:::smithi17529310-14:head [rollback 19] snapc 27=[27,26,24,22,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.304773357999998, 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.011011909, 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 747 2024-09-15T01:33:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8299999999999995e-07 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078479+0000", 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5549999999999996e-06 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078489+0000", 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0339999999999999e-05 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078495+0000", 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4999999999999999e-06 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078537+0000", 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2194999999999998e-05 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078544+0000", 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0759999999999997e-06 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.084273+0000", 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057284170000000004 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.084275+0000", 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4049999999999998e-06 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087195+0000", 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.002919946 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087315+0000", 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000119883 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087944+0000", 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00062966699999999999 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088093+0000", 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000148531 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088104+0000", 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1381e-05 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088151+0000", 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6495000000000001e-05 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088666+0000", 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051494400000000004 2024-09-15T01:33:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.089420+0000", 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00075379099999999997 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.089485+0000", 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5900000000000003e-05 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 81/76 ECSubWriteReply(tid=313, last_complete=81'63, committed=1, applied=1))", 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:23.083561+0000", 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.299685484999999, 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87779885599999996, 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083561+0000", 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083561+0000", 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083562+0000", 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5400000000000002e-07 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083564+0000", 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4739999999999999e-06 2024-09-15T01:33:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.960927+0000", 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87736222600000002 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.960933+0000", 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3550000000000001e-06 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.961015+0000", 2024-09-15T01:33:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1646999999999999e-05 2024-09-15T01:33:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.961360+0000", 2024-09-15T01:33:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0003457 2024-09-15T01:33:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 87/31 ECSubWriteReply(tid=365, last_complete=87'192, committed=1, applied=1))", 2024-09-15T01:33:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:29.144736+0000", 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.238511302, 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.52162740600000002, 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.144736+0000", 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.144736+0000", 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.144736+0000", 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2900000000000004e-07 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.144740+0000", 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4300000000000002e-06 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.665922+0000", 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.52118211400000003 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.665927+0000", 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3469999999999998e-06 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.666003+0000", 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6185000000000003e-05 2024-09-15T01:33:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.666363+0000", 2024-09-15T01:33:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035980099999999998 2024-09-15T01:33:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:35.489 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:35.490 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:35.491 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:35.491 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:35.492 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:35.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-15T01:33:35.492 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:35.493 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:35.493 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:35.493 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:35.494 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:35.494 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:35.495 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:35.489+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:35.513 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:35.513 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:35.533 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:35.533 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:35.551 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:35.560 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 9] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-15T01:33:35.560 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:33:35.560 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-15T01:33:35.560 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T01:33:35.560 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T01:33:35.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-15T01:33:35.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:35.571+0000 7f72df7fe640 0 -- 172.21.15.42:0/704102323 >> [v2:172.21.15.115:6824/28511,v1:172.21.15.115:6825/28511] conn(0x7f72e0192040 msgr2=0x7f72e0194420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:35.580 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:35.591 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:35.589+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:35.599 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:35.600 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:35.619 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:35.619 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:35.629 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:35.629 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:35.637 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-15T01:33:35.663 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:33:35.663 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T01:33:35.663 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:33:35.713 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:35.711+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:35.734 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:35.734 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:35.742 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-15T01:33:35.793 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:35.792+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:35.839 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:35.839 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:35.847 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:33:35.893 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:35.892+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:35.950 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:35.950 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:35.959 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:33:36.000 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:35.998+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:36.095 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:36.094+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:36.097 INFO:tasks.rados.rados.0.smithi175.stdout:356: done (3 left) 2024-09-15T01:33:36.097 INFO:tasks.rados.rados.0.smithi175.stdout:357: done (2 left) 2024-09-15T01:33:36.097 INFO:tasks.rados.rados.0.smithi175.stdout:358: done (1 left) 2024-09-15T01:33:36.097 INFO:tasks.rados.rados.0.smithi175.stdout:360: done (0 left) 2024-09-15T01:33:36.098 INFO:tasks.rados.rados.0.smithi175.stdout:361: snap_remove snap 17 2024-09-15T01:33:36.101 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:36.101 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:36.106 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:33:36.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:33:36.197 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:36.195+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:36.254 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:36.254 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:36.263 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:33:36.298 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:36.296+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:36.364 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:36.364 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:36.373 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:33:36.417 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:36.416+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:36.474 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:36.474 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:36.482 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:33:36.500 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:36.498+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:36.601 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:36.599+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:36.624 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:36.624 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:36.632 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:33:36.702 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:36.701+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:36.733 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:36.733 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:36.742 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:33:36.804 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:36.802+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:36.842 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:36.842 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:36.850 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:33:36.905 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:36.903+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:36.950 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:36.950 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:36.959 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-15T01:33:37.006 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:37.004+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:37.060 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:37.060 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:37.068 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-15T01:33:37.096 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-15T01:33:37.104 INFO:tasks.rados.rados.0.smithi175.stdout:361: done (0 left) 2024-09-15T01:33:37.104 INFO:tasks.rados.rados.0.smithi175.stdout:362: read oid 9 snap -1 2024-09-15T01:33:37.104 INFO:tasks.rados.rados.0.smithi175.stdout:362: expect deleted 2024-09-15T01:33:37.104 INFO:tasks.rados.rados.0.smithi175.stdout:363: read oid 2 snap -1 2024-09-15T01:33:37.104 INFO:tasks.rados.rados.0.smithi175.stdout:363: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-15T01:33:37.105 INFO:tasks.rados.rados.0.smithi175.stdout:364: read oid 39 snap -1 2024-09-15T01:33:37.105 INFO:tasks.rados.rados.0.smithi175.stdout:364: expect (ObjNum 139 snap 29 seq_num 139) 2024-09-15T01:33:37.105 INFO:tasks.rados.rados.0.smithi175.stdout:365: snap_remove snap 23 2024-09-15T01:33:37.108 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:37.105+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:37.204 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:37.204 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:37.209 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:37.206+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:37.211 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-15T01:33:37.309 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:37.307+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:37.312 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:37.312 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:37.321 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:33:37.410 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:37.409+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:37.429 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:37.429 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:37.439 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:33:37.511 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:37.510+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:37.537 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:37.537 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:37.546 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:33:37.613 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:37.612+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:37.631 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:33:37.642 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:37.642 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:37.651 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:33:37.714 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:37.711+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:37.748 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:37.748 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:37.756 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:33:37.759 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:37.759 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:37.778 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:37.778 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:37.798 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:37.798 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:37.815 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:37.812+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:37.820 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:37.821 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:37.852 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:37.852 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:37.861 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:33:37.916 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:37.915+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:37.957 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:37.957 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:37.965 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:33:38.017 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:38.015+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:38.102 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:38.103 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:38.105 INFO:tasks.rados.rados.0.smithi175.stdout:365: done (3 left) 2024-09-15T01:33:38.105 INFO:tasks.rados.rados.0.smithi175.stdout:366: read oid 45 snap 12 2024-09-15T01:33:38.105 INFO:tasks.rados.rados.0.smithi175.stdout:366: expect (ObjNum 68 snap 3 seq_num 68) 2024-09-15T01:33:38.105 INFO:tasks.rados.rados.0.smithi175.stdout:367: snap_remove snap 29 2024-09-15T01:33:38.116 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:33:38.117 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:38.116+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:38.214 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:38.214 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:38.218 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:38.217+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:38.226 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:33:38.318 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:38.316+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:38.323 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:38.323 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:38.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-15T01:33:38.419 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:38.418+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:38.429 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:38.430 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:38.436 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-15T01:33:38.520 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:38.518+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:38.532 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:38.532 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:38.541 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-15T01:33:38.621 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:38.619+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:38.636 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:38.636 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:38.644 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:33:38.722 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:38.719+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:38.753 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:39.092 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:39.093 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:39.093 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:39.094 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:33:39.095 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:39.091+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:39.109 INFO:tasks.rados.rados.0.smithi175.stdout:367: done (4 left) 2024-09-15T01:33:39.110 INFO:tasks.rados.rados.0.smithi175.stdout:368: snap_remove snap 28 2024-09-15T01:33:39.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:33:39.194 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:39.193+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:39.203 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:39.203 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:39.203 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:39.203 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:39.203 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:39.212 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:33:39.295 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:39.294+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:39.311 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:39.311 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:39.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:39.311 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:39.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.311 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=152, reqid=osd.13.0:152, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:39.311 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.243665+0000", 2024-09-15T01:33:39.311 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.064653608999997, 2024-09-15T01:33:39.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019478882999999999, 2024-09-15T01:33:39.311 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243665+0000", 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243665+0000", 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243666+0000", 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.34e-07 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243686+0000", 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9830000000000002e-05 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243778+0000", 2024-09-15T01:33:39.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2367999999999994e-05 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243802+0000", 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3646e-05 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243862+0000", 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0579999999999999e-05 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243876+0000", 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4025e-05 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245801+0000", 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0019244399999999999 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246115+0000", 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00031487699999999997 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248315+0000", 2024-09-15T01:33:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021992140000000001 2024-09-15T01:33:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248329+0000", 2024-09-15T01:33:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4450000000000001e-05 2024-09-15T01:33:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263144+0000", 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014815218999999999 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.061351006999999, 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019581815999999998, 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246969+0000", 2024-09-15T01:33:39.318 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7000000000000003e-07 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247008+0000", 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9354999999999998e-05 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247051+0000", 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2849000000000003e-05 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247060+0000", 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4589999999999998e-06 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248680+0000", 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001620058 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248690+0000", 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8660000000000001e-06 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248945+0000", 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025509900000000001 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251105+0000", 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021604290000000002 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251177+0000", 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1680999999999994e-05 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251189+0000", 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2661e-05 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266550+0000", 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015360389 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=28, reqid=osd.8.0:28, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:39.322 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.058169135999997, 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022199446000000001, 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.323 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4700000000000001e-07 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250167+0000", 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7384000000000002e-05 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.324 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250202+0000", 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4912e-05 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250210+0000", 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6029999999999999e-06 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250258+0000", 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8109999999999998e-05 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250265+0000", 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4400000000000002e-06 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250466+0000", 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000201227 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251113+0000", 2024-09-15T01:33:39.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00064769299999999997 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251179+0000", 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5895999999999994e-05 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251196+0000", 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6810999999999999e-05 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272349+0000", 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021153222999999999 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.056815008000001, 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023186075, 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.328 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.329 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.329 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.329 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:33:39.329 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.329 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.329 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.329 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.329 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:33:39.329 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.329 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.329 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.329 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251505+0000", 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.09e-07 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251531+0000", 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5834999999999999e-05 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251574+0000", 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2879000000000001e-05 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251580+0000", 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8460000000000002e-06 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251639+0000", 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8331000000000001e-05 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251649+0000", 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9930000000000007e-06 2024-09-15T01:33:39.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251890+0000", 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00024173400000000001 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251923+0000", 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2607999999999997e-05 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251985+0000", 2024-09-15T01:33:39.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2238999999999998e-05 2024-09-15T01:33:39.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.333 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.333 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251998+0000", 2024-09-15T01:33:39.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2979999999999999e-05 2024-09-15T01:33:39.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.333 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.333 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274690+0000", 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022691821000000001 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s1 44/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=44'26))", 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.055598009999997, 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019696631999999999, 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252722+0000", 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3099999999999996e-07 2024-09-15T01:33:39.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252734+0000", 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1576999999999999e-05 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252780+0000", 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6820999999999998e-05 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252788+0000", 2024-09-15T01:33:39.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2710000000000003e-06 2024-09-15T01:33:39.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.337 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252848+0000", 2024-09-15T01:33:39.337 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0718999999999999e-05 2024-09-15T01:33:39.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.337 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252859+0000", 2024-09-15T01:33:39.337 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0342e-05 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252960+0000", 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000101741 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252970+0000", 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0008e-05 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253967+0000", 2024-09-15T01:33:39.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00099612599999999996 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253987+0000", 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0064e-05 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018431032 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=154, reqid=client.4477.0:282, at_version=44'19, trim_to=0'0, roll_forward_to=44'17))", 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.051586350000001, 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023652260000000001, 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.340 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256734+0000", 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3900000000000004e-07 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256754+0000", 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0279e-05 2024-09-15T01:33:39.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256785+0000", 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0703000000000001e-05 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256798+0000", 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3606e-05 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256843+0000", 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5065000000000003e-05 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256851+0000", 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5580000000000003e-06 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257013+0000", 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016264 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257339+0000", 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032564200000000001 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257402+0000", 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3162000000000003e-05 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258083+0000", 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00068126099999999995 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280385+0000", 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022301504999999999 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=85, reqid=osd.11.0:84, at_version=0'0, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.051262565999998, 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020078747000000001, 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.345 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.346 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257057+0000", 2024-09-15T01:33:39.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5000000000000002e-07 2024-09-15T01:33:39.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.347 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.347 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257067+0000", 2024-09-15T01:33:39.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5540000000000003e-06 2024-09-15T01:33:39.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.347 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.347 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257097+0000", 2024-09-15T01:33:39.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9892000000000001e-05 2024-09-15T01:33:39.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257102+0000", 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3360000000000002e-06 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257155+0000", 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3313000000000003e-05 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257164+0000", 2024-09-15T01:33:39.348 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1060000000000006e-06 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257257+0000", 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2678999999999998e-05 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257341+0000", 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3459000000000002e-05 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.349 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257403+0000", 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2477000000000005e-05 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257420+0000", 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7170999999999999e-05 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277135+0000", 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019715010000000002 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.350 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.351 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.351 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.351 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=30, reqid=osd.8.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:39.351 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:39.351 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.050995295, 2024-09-15T01:33:39.351 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020096306000000001, 2024-09-15T01:33:39.351 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.351 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.351 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.351 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.351 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.351 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:39.352 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.352 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.352 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.352 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:39.352 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.352 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.352 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.352 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:39.352 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3099999999999997e-07 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257335+0000", 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0727000000000001e-05 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257371+0000", 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5583999999999997e-05 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.353 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257378+0000", 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.754e-06 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257427+0000", 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9048999999999997e-05 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257475+0000", 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8140999999999999e-05 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.354 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257605+0000", 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012981700000000001 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257640+0000", 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.557e-05 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257674+0000", 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4215999999999999e-05 2024-09-15T01:33:39.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257692+0000", 2024-09-15T01:33:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7988000000000001e-05 2024-09-15T01:33:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277420+0000", 2024-09-15T01:33:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019727728999999999 2024-09-15T01:33:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=86, reqid=osd.11.0:86, at_version=44'42, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.050404041, 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022807320999999998, 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:33:39.357 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257916+0000", 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.358 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257951+0000", 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5754999999999999e-05 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257982+0000", 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0732000000000003e-05 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257987+0000", 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1900000000000003e-06 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.359 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258036+0000", 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8547999999999997e-05 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258047+0000", 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1082e-05 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258512+0000", 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00046539599999999997 2024-09-15T01:33:39.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258527+0000", 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5121e-05 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258568+0000", 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0305000000000001e-05 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258581+0000", 2024-09-15T01:33:39.362 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3434999999999999e-05 2024-09-15T01:33:39.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.362 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.362 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.362 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280722+0000", 2024-09-15T01:33:39.362 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022141137000000002 2024-09-15T01:33:39.362 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.362 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.362 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.362 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.362 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=155, reqid=client.4477.0:282, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.049479243999997, 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021774615000000001, 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258841+0000", 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5899999999999995e-07 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258853+0000", 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2115e-05 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258889+0000", 2024-09-15T01:33:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6007000000000003e-05 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258896+0000", 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258958+0000", 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2242999999999993e-05 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.365 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258966+0000", 2024-09-15T01:33:39.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2370000000000003e-06 2024-09-15T01:33:39.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259070+0000", 2024-09-15T01:33:39.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000104171 2024-09-15T01:33:39.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259104+0000", 2024-09-15T01:33:39.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4569000000000001e-05 2024-09-15T01:33:39.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259140+0000", 2024-09-15T01:33:39.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5698000000000003e-05 2024-09-15T01:33:39.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259155+0000", 2024-09-15T01:33:39.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4541e-05 2024-09-15T01:33:39.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280614+0000", 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021459865000000002 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.049049869000001, 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027513342999999999, 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.368 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259270+0000", 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.33e-07 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259295+0000", 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4413000000000002e-05 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259335+0000", 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0154999999999998e-05 2024-09-15T01:33:39.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259344+0000", 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8890000000000008e-06 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259414+0000", 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9929000000000001e-05 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259423+0000", 2024-09-15T01:33:39.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0289999999999994e-06 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263376+0000", 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003953036 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263393+0000", 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7444000000000001e-05 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263481+0000", 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8251999999999995e-05 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266784+0000", 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003303074 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286783+0000", 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019998189 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.373 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=31, reqid=client.4477.0:288, at_version=44'22, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.044282498000001, 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022720738000000001, 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6599999999999995e-07 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264398+0000", 2024-09-15T01:33:39.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00036057299999999999 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264436+0000", 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7968e-05 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264444+0000", 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8949999999999998e-06 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264489+0000", 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4954999999999999e-05 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264499+0000", 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0574000000000001e-05 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266178+0000", 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001678543 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266188+0000", 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0624e-05 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268728+0000", 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025397240000000001 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268912+0000", 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018415400000000001 2024-09-15T01:33:39.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286757+0000", 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.017844961999999999 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=32, reqid=client.4477.0:289, at_version=44'23, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.041083059999998, 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019719520000000001, 2024-09-15T01:33:39.379 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267237+0000", 2024-09-15T01:33:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3099999999999998e-07 2024-09-15T01:33:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267495+0000", 2024-09-15T01:33:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025857700000000001 2024-09-15T01:33:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267528+0000", 2024-09-15T01:33:39.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2617999999999998e-05 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267534+0000", 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0159999999999997e-06 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267584+0000", 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0176999999999997e-05 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267594+0000", 2024-09-15T01:33:39.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0161e-05 2024-09-15T01:33:39.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267768+0000", 2024-09-15T01:33:39.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017420000000000001 2024-09-15T01:33:39.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268732+0000", 2024-09-15T01:33:39.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00096413399999999995 2024-09-15T01:33:39.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268818+0000", 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5583999999999999e-05 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268943+0000", 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012452399999999999 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286956+0000", 2024-09-15T01:33:39.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018012997999999999 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=91, reqid=osd.11.0:87, at_version=0'0, trim_to=0'0, roll_forward_to=44'44))", 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.040990616000002, 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019815672999999999, 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.385 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267329+0000", 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7500000000000001e-07 2024-09-15T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267339+0000", 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0351e-05 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267385+0000", 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6008999999999999e-05 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267393+0000", 2024-09-15T01:33:39.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7810000000000005e-06 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267443+0000", 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0139000000000002e-05 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267454+0000", 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1012999999999999e-05 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267585+0000", 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013078699999999999 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268732+0000", 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001146768 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268817+0000", 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5724999999999997e-05 2024-09-15T01:33:39.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268838+0000", 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0954e-05 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.287144+0000", 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018305770999999998 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=32, reqid=client.4477.0:294, at_version=44'23, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.038016403, 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021391184000000001, 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270304+0000", 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7499999999999998e-07 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270326+0000", 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2821999999999999e-05 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270366+0000", 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9708e-05 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270374+0000", 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6709999999999999e-06 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270433+0000", 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9274999999999997e-05 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270444+0000", 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.131e-05 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271974+0000", 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001529996 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271986+0000", 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1443e-05 2024-09-15T01:33:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272180+0000", 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019405299999999999 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272366+0000", 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018627699999999999 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.291694+0000", 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019327654 2024-09-15T01:33:39.395 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.037419724999999, 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024535670999999998, 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270900+0000", 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.23e-07 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270920+0000", 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.022e-05 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270953+0000", 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3136000000000003e-05 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270960+0000", 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4280000000000001e-06 2024-09-15T01:33:39.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272041+0000", 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010816440000000001 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272050+0000", 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5269999999999998e-06 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272149+0000", 2024-09-15T01:33:39.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9464999999999997e-05 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272182+0000", 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2792e-05 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272209+0000", 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6409000000000001e-05 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272435+0000", 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000226691 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295435+0000", 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022999636 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=34, reqid=client.4477.0:296, at_version=44'25, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:39.401 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.036853618000002, 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024251137999999998, 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:33:39.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271466+0000", 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8899999999999998e-07 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271488+0000", 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1875999999999999e-05 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271522+0000", 2024-09-15T01:33:39.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4041999999999997e-05 2024-09-15T01:33:39.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271529+0000", 2024-09-15T01:33:39.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4620000000000001e-06 2024-09-15T01:33:39.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272189+0000", 2024-09-15T01:33:39.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066068699999999997 2024-09-15T01:33:39.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272198+0000", 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2900000000000002e-06 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272284+0000", 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6638999999999995e-05 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272322+0000", 2024-09-15T01:33:39.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7125000000000001e-05 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272357+0000", 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5626000000000001e-05 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272458+0000", 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000100343 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295717+0000", 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023259058999999999 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=35, reqid=client.4477.0:291, at_version=0'0, trim_to=0'0, roll_forward_to=44'25))", 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.035246481000001, 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022393413000000001, 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0999999999999998e-07 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273083+0000", 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0015e-05 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273110+0000", 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6798999999999999e-05 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273117+0000", 2024-09-15T01:33:39.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5379999999999999e-06 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273155+0000", 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8563999999999999e-05 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273163+0000", 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4939999999999997e-06 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273248+0000", 2024-09-15T01:33:39.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5290000000000005e-05 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273254+0000", 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3149999999999997e-06 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273288+0000", 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3958000000000003e-05 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273303+0000", 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4731e-05 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295466+0000", 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022162998999999999 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 54/31 ECSubWrite(tid=63, reqid=client.4477.0:459, at_version=54'49, trim_to=0'0, roll_forward_to=54'46))", 2024-09-15T01:33:39.412 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.094073+0000", 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 43.214245591000001, 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.82711023299999997, 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094073+0000", 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094073+0000", 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094074+0000", 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4099999999999999e-07 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094105+0000", 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1087000000000002e-05 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920807+0000", 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.82670179300000002 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920813+0000", 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8100000000000003e-06 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920878+0000", 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4870999999999996e-05 2024-09-15T01:33:39.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920885+0000", 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3860000000000001e-06 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920975+0000", 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9797999999999996e-05 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920981+0000", 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2419999999999997e-06 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.921033+0000", 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1811999999999998e-05 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.921048+0000", 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5135e-05 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.921184+0000", 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013575800000000001 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpRead(3.fs1 69/31 ECSubRead(tid=220, to_read={3:f73c1a72:::smithi17529310-29:head=0,4096,0}, subchunks={3:f73c1a72:::smithi17529310-29:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:11.104731+0000", 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 28.203588328999999, 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.77828088100000004, 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.104731+0000", 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.104731+0000", 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.104731+0000", 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8699999999999999e-07 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.104736+0000", 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7659999999999998e-06 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.882809+0000", 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.77807284600000004 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.882814+0000", 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8889999999999999e-06 2024-09-15T01:33:39.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.882901+0000", 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6193000000000006e-05 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.883012+0000", 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000111 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.421 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.422 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:39.423 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:33:39.423 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:39.421+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:39.508 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:39.508 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: filestore_inject_stall = '' (not observed, change may require restart) osd_enable_op_tracker = '' 2024-09-15T01:33:39.523 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:39.521+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:39.527 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:39.527 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:39.546 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:39.546 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:39.563 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:39.563 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:39.566 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:39.566 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:39.571 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:33:39.584 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:39.602 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:39.603 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:39.622 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:39.624 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:39.623+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:39.641 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:39.641 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:39.660 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:39.680 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:39.698 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:39.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:39.697+0000 7fe35ed9d640 0 -- 172.21.15.42:0/677664650 >> [v2:172.21.15.175:6818/28838,v1:172.21.15.175:6819/28838] conn(0x7fe360195c60 msgr2=0x7fe360198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:39.709 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:39.709 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:39.709 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:39.709 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:39.709 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:39.717 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:33:39.724 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:39.722+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:39.818 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:39.818 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:39.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:39.818 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:39.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.574518163, 2024-09-15T01:33:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022664864999999999, 2024-09-15T01:33:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.16e-07 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241155+0000", 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016715400000000001 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241334+0000", 2024-09-15T01:33:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017883600000000001 2024-09-15T01:33:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241342+0000", 2024-09-15T01:33:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1519999999999992e-06 2024-09-15T01:33:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241446+0000", 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010419 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241468+0000", 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1747e-05 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.244959+0000", 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0034911600000000001 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.244970+0000", 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1131000000000001e-05 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245013+0000", 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3214000000000003e-05 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245498+0000", 2024-09-15T01:33:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00048501499999999999 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263652+0000", 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018154050000000001 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.574320202999999, 2024-09-15T01:33:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022656372000000001, 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9399999999999999e-07 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241203+0000", 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7152000000000001e-05 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241707+0000", 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050421299999999999 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241714+0000", 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4409999999999998e-06 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241773+0000", 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9762000000000001e-05 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241783+0000", 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6860000000000001e-06 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245035+0000", 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032521130000000001 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245184+0000", 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014925800000000001 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245769+0000", 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058491399999999996 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246140+0000", 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037060199999999999 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263842+0000", 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.017702037 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.573103085, 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034719353000000001, 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1600000000000001e-07 2024-09-15T01:33:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242427+0000", 2024-09-15T01:33:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4022000000000001e-05 2024-09-15T01:33:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242460+0000", 2024-09-15T01:33:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2496000000000002e-05 2024-09-15T01:33:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242476+0000", 2024-09-15T01:33:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6503000000000001e-05 2024-09-15T01:33:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245083+0000", 2024-09-15T01:33:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.002606531 2024-09-15T01:33:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245092+0000", 2024-09-15T01:33:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7679999999999997e-06 2024-09-15T01:33:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245298+0000", 2024-09-15T01:33:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000205971 2024-09-15T01:33:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245772+0000", 2024-09-15T01:33:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00047452200000000001 2024-09-15T01:33:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245817+0000", 2024-09-15T01:33:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4474999999999997e-05 2024-09-15T01:33:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246171+0000", 2024-09-15T01:33:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035433200000000001 2024-09-15T01:33:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277122+0000", 2024-09-15T01:33:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030951117 2024-09-15T01:33:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.572479553999997, 2024-09-15T01:33:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031640535999999997, 2024-09-15T01:33:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.49e-07 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243044+0000", 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7946999999999999e-05 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243140+0000", 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5311999999999998e-05 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243147+0000", 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2069999999999998e-06 2024-09-15T01:33:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245010+0000", 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018629230000000001 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245020+0000", 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0774999999999999e-05 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245472+0000", 2024-09-15T01:33:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045200900000000002 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245773+0000", 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00030029999999999998 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245817+0000", 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4367999999999999e-05 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246317+0000", 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050021500000000003 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274667+0000", 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028349330999999998 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.568215508000002, 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027771575999999999, 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247291+0000", 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7700000000000004e-07 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247302+0000", 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1246e-05 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247333+0000", 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1179000000000001e-05 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247341+0000", 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.571e-06 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247388+0000", 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6876999999999999e-05 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247395+0000", 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3359999999999997e-06 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247478+0000", 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2860000000000003e-05 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247487+0000", 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9029999999999993e-06 2024-09-15T01:33:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247526+0000", 2024-09-15T01:33:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8829999999999999e-05 2024-09-15T01:33:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247546+0000", 2024-09-15T01:33:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0551999999999999e-05 2024-09-15T01:33:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275062+0000", 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027515444999999999 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as5 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.567992687999997, 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027915809999999999, 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247514+0000", 2024-09-15T01:33:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9899999999999999e-07 2024-09-15T01:33:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247524+0000", 2024-09-15T01:33:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0396e-05 2024-09-15T01:33:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247555+0000", 2024-09-15T01:33:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0414000000000001e-05 2024-09-15T01:33:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247564+0000", 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1900000000000001e-06 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247616+0000", 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1788e-05 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247628+0000", 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.288e-05 2024-09-15T01:33:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247763+0000", 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013473700000000001 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247772+0000", 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3429999999999996e-06 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247804+0000", 2024-09-15T01:33:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2533999999999997e-05 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247817+0000", 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2694000000000001e-05 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275429+0000", 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027612035 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s5 44/31 ECSubWrite(tid=65, reqid=osd.14.0:65, at_version=44'26, trim_to=0'0, roll_forward_to=44'24))", 2024-09-15T01:33:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.565947342000001, 2024-09-15T01:33:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.026123550999999998, 2024-09-15T01:33:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249559+0000", 2024-09-15T01:33:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3200000000000003e-07 2024-09-15T01:33:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249584+0000", 2024-09-15T01:33:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4284999999999999e-05 2024-09-15T01:33:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249633+0000", 2024-09-15T01:33:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9526e-05 2024-09-15T01:33:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249641+0000", 2024-09-15T01:33:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2239999999999995e-06 2024-09-15T01:33:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249695+0000", 2024-09-15T01:33:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3674999999999997e-05 2024-09-15T01:33:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249705+0000", 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8300000000000008e-06 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251518+0000", 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018128409999999999 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251549+0000", 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0948999999999998e-05 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251591+0000", 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2515999999999997e-05 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251612+0000", 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0494000000000001e-05 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275682+0000", 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024070279 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s5 44/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=44'26))", 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.562753632000003, 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022959355000000001, 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252753+0000", 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7499999999999998e-07 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252764+0000", 2024-09-15T01:33:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1123e-05 2024-09-15T01:33:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252899+0000", 2024-09-15T01:33:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013444100000000001 2024-09-15T01:33:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252906+0000", 2024-09-15T01:33:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8650000000000001e-06 2024-09-15T01:33:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252959+0000", 2024-09-15T01:33:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2698000000000003e-05 2024-09-15T01:33:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252968+0000", 2024-09-15T01:33:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2140000000000002e-06 2024-09-15T01:33:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253058+0000", 2024-09-15T01:33:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9926999999999995e-05 2024-09-15T01:33:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253066+0000", 2024-09-15T01:33:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6620000000000005e-06 2024-09-15T01:33:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253105+0000", 2024-09-15T01:33:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8875999999999998e-05 2024-09-15T01:33:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253117+0000", 2024-09-15T01:33:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2405000000000001e-05 2024-09-15T01:33:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275711+0000", 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022594169000000001 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=60, reqid=client.4477.0:306, at_version=44'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.544677231999998, 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024129333999999999, 2024-09-15T01:33:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270829+0000", 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270845+0000", 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6013999999999999e-05 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270871+0000", 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6140999999999999e-05 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270877+0000", 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7989999999999999e-06 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270947+0000", 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9480000000000006e-05 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270959+0000", 2024-09-15T01:33:39.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2395000000000001e-05 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274484+0000", 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003524976 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274715+0000", 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000230595 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274806+0000", 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1130000000000003e-05 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274847+0000", 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1473000000000001e-05 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.294958+0000", 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020110711 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.878 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.544599222000002, 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020359914999999999, 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270907+0000", 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4200000000000001e-07 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270919+0000", 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1761e-05 2024-09-15T01:33:39.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270936+0000", 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7761000000000002e-05 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270953+0000", 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6858e-05 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272042+0000", 2024-09-15T01:33:39.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010882769999999999 2024-09-15T01:33:39.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272051+0000", 2024-09-15T01:33:39.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8400000000000007e-06 2024-09-15T01:33:39.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272145+0000", 2024-09-15T01:33:39.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3639000000000003e-05 2024-09-15T01:33:39.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272153+0000", 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6240000000000002e-06 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272221+0000", 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8281999999999994e-05 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272241+0000", 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9542e-05 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.291266+0000", 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019025889000000001 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:39.884 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.543087976000002, 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.025858174000000001, 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:39.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272446+0000", 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7114000000000002e-05 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272489+0000", 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3711999999999998e-05 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272498+0000", 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9169999999999995e-06 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274537+0000", 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0020384190000000001 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274545+0000", 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1999999999999994e-06 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274659+0000", 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011420499999999999 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274716+0000", 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7114999999999998e-05 2024-09-15T01:33:39.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274807+0000", 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0346999999999994e-05 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274883+0000", 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.695e-05 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298276+0000", 2024-09-15T01:33:39.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023392452000000001 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=62, reqid=client.4477.0:308, at_version=44'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.542040825999997, 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024943603000000002, 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.37e-07 2024-09-15T01:33:39.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273478+0000", 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2768e-05 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273502+0000", 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3920000000000001e-05 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273508+0000", 2024-09-15T01:33:39.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0909999999999998e-06 2024-09-15T01:33:39.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274699+0000", 2024-09-15T01:33:39.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011909240000000001 2024-09-15T01:33:39.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274707+0000", 2024-09-15T01:33:39.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5429999999999996e-06 2024-09-15T01:33:39.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274788+0000", 2024-09-15T01:33:39.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1804000000000005e-05 2024-09-15T01:33:39.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274811+0000", 2024-09-15T01:33:39.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2379e-05 2024-09-15T01:33:39.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274902+0000", 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1031000000000003e-05 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274910+0000", 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8359999999999992e-06 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298408+0000", 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023498769999999999 2024-09-15T01:33:39.896 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.897 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.897 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.897 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=92, reqid=client.4477.0:309, at_version=44'31, trim_to=0'0, roll_forward_to=44'28))", 2024-09-15T01:33:39.897 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:39.897 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.540941347999997, 2024-09-15T01:33:39.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024021724000000001, 2024-09-15T01:33:39.897 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.897 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.897 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274565+0000", 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9299999999999996e-07 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275016+0000", 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045075999999999997 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275077+0000", 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0582999999999999e-05 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275084+0000", 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1289999999999998e-06 2024-09-15T01:33:39.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275144+0000", 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0566000000000003e-05 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275153+0000", 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4849999999999992e-06 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275302+0000", 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000149689 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275327+0000", 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4337000000000001e-05 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277020+0000", 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001693153 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277074+0000", 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4122999999999997e-05 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298586+0000", 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021512006 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 44/31 ECSubWrite(tid=35, reqid=client.4477.0:296, at_version=0'0, trim_to=0'0, roll_forward_to=44'25))", 2024-09-15T01:33:39.902 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.537399374000003, 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020797596000000002, 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:39.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278107+0000", 2024-09-15T01:33:39.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8199999999999998e-07 2024-09-15T01:33:39.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278119+0000", 2024-09-15T01:33:39.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1567e-05 2024-09-15T01:33:39.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278155+0000", 2024-09-15T01:33:39.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6560999999999998e-05 2024-09-15T01:33:39.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278163+0000", 2024-09-15T01:33:39.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2819999999999999e-06 2024-09-15T01:33:39.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278209+0000", 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6146999999999997e-05 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278219+0000", 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0699999999999999e-05 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278342+0000", 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000122061 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278350+0000", 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4460000000000001e-06 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280088+0000", 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0017376309999999999 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280107+0000", 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.986e-05 2024-09-15T01:33:39.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298904+0000", 2024-09-15T01:33:39.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018796459000000001 2024-09-15T01:33:39.908 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.908 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.908 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.908 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=93, reqid=client.4477.0:310, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:39.908 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:39.909 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.536691371000003, 2024-09-15T01:33:39.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020226037999999998, 2024-09-15T01:33:39.909 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.909 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.909 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:39.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278815+0000", 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6000000000000003e-07 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279204+0000", 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000388413 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279228+0000", 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4674000000000001e-05 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279233+0000", 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7849999999999999e-06 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279272+0000", 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9502999999999998e-05 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279278+0000", 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0079999999999999e-06 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279401+0000", 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012238799999999999 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280091+0000", 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00069018900000000002 2024-09-15T01:33:39.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280213+0000", 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012227700000000001 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280231+0000", 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7408999999999999e-05 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.299040+0000", 2024-09-15T01:33:39.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018809632 2024-09-15T01:33:39.914 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.914 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.914 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.914 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.5s6 44/31 ECSubWrite(tid=15, reqid=client.4477.0:314, at_version=44'8, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:39.914 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:39.914 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.536141915000002, 2024-09-15T01:33:39.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.026604415999999999, 2024-09-15T01:33:39.914 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.914 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.914 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:39.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279386+0000", 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.211e-05 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279421+0000", 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4523000000000001e-05 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279428+0000", 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4420000000000001e-06 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279474+0000", 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5995000000000003e-05 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279480+0000", 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.105e-06 2024-09-15T01:33:39.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279645+0000", 2024-09-15T01:33:39.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016430199999999999 2024-09-15T01:33:39.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280092+0000", 2024-09-15T01:33:39.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044707999999999998 2024-09-15T01:33:39.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280214+0000", 2024-09-15T01:33:39.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012248399999999999 2024-09-15T01:33:39.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280260+0000", 2024-09-15T01:33:39.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5785000000000003e-05 2024-09-15T01:33:39.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.305968+0000", 2024-09-15T01:33:39.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.025708106000000001 2024-09-15T01:33:39.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.920 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.920 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.5s6 44/31 ECSubWrite(tid=16, reqid=client.4477.0:314, at_version=0'0, trim_to=0'0, roll_forward_to=44'8))", 2024-09-15T01:33:39.920 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:39.920 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.533487340999997, 2024-09-15T01:33:39.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024148095000000001, 2024-09-15T01:33:39.921 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.921 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.921 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:39.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:39.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282019+0000", 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282028+0000", 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0969999999999995e-06 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282054+0000", 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5897e-05 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282059+0000", 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7849999999999999e-06 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282101+0000", 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1801000000000002e-05 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282108+0000", 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0650000000000001e-06 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282171+0000", 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2834000000000002e-05 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282844+0000", 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00067343600000000002 2024-09-15T01:33:39.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282915+0000", 2024-09-15T01:33:39.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1242999999999995e-05 2024-09-15T01:33:39.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282933+0000", 2024-09-15T01:33:39.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8136000000000001e-05 2024-09-15T01:33:39.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306166+0000", 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023233052000000001 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=63, reqid=client.4477.0:308, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.531903708999998, 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022584495, 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.926 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283603+0000", 2024-09-15T01:33:39.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5899999999999995e-07 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283618+0000", 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5600999999999999e-05 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283648+0000", 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9295e-05 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283653+0000", 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1050000000000003e-06 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283703+0000", 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9883999999999999e-05 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283712+0000", 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3740000000000004e-06 2024-09-15T01:33:39.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283859+0000", 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000146959 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283869+0000", 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0295999999999999e-05 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283906+0000", 2024-09-15T01:33:39.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6534e-05 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283922+0000", 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6560000000000001e-05 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306187+0000", 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022264128000000001 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 47/38 ECSubWrite(tid=76, reqid=client.4477.0:376, at_version=47'25, trim_to=0'0, roll_forward_to=47'22))", 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 50.451699546, 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.61852187999999997, 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363807+0000", 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5799999999999999e-07 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363818+0000", 2024-09-15T01:33:39.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1729000000000001e-05 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981818+0000", 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.61799953399999996 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981826+0000", 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1049999999999995e-06 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981893+0000", 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7229000000000003e-05 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981905+0000", 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1527e-05 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982064+0000", 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000159147 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982078+0000", 2024-09-15T01:33:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:33:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982122+0000", 2024-09-15T01:33:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4354000000000003e-05 2024-09-15T01:33:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982140+0000", 2024-09-15T01:33:39.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8315e-05 2024-09-15T01:33:39.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982328+0000", 2024-09-15T01:33:39.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018781200000000001 2024-09-15T01:33:39.937 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.937 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.937 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.938 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 54/52 ECSubWrite(tid=119, reqid=client.4477.0:500, at_version=54'41, trim_to=0'0, roll_forward_to=54'38))", 2024-09-15T01:33:39.938 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:39.938 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 43.604204514999999, 2024-09-15T01:33:39.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.27782579699999999, 2024-09-15T01:33:39.938 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:39.938 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:39.938 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:39.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:39.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:39.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:39.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:39.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:39.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:39.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211302+0000", 2024-09-15T01:33:39.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.89e-07 2024-09-15T01:33:39.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211313+0000", 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1521e-05 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488807+0000", 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.27749379200000002 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:39.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488811+0000", 2024-09-15T01:33:39.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2889999999999999e-06 2024-09-15T01:33:39.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:39.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488867+0000", 2024-09-15T01:33:39.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5322000000000003e-05 2024-09-15T01:33:39.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488874+0000", 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9659999999999996e-06 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488953+0000", 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8591999999999996e-05 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488959+0000", 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0000000000000002e-06 2024-09-15T01:33:39.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488993+0000", 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4326999999999998e-05 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.489009+0000", 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6015000000000002e-05 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.489127+0000", 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117584 2024-09-15T01:33:39.943 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.944 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:39.944 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:39.944 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:39.945 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:39.946 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:33:39.947 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:39.947 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:39.943+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:40.044 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:40.044 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:40.044 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:40.044 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:40.046 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:40.045+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:40.052 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:33:40.110 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-15T01:33:40.113 INFO:tasks.rados.rados.0.smithi175.stdout:368: done (4 left) 2024-09-15T01:33:40.113 INFO:tasks.rados.rados.0.smithi175.stdout:369: copy_from oid 22 from oid 15 current snap is 33 2024-09-15T01:33:40.113 INFO:tasks.rados.rados.0.smithi175.stdout:370: rollback oid 1 current snap is 33 2024-09-15T01:33:40.113 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 1 to 31 2024-09-15T01:33:40.113 INFO:tasks.rados.rados.0.smithi175.stdout:371: read oid 48 snap 25 2024-09-15T01:33:40.114 INFO:tasks.rados.rados.0.smithi175.stdout:371: expect (ObjNum 84 snap 7 seq_num 84) 2024-09-15T01:33:40.114 INFO:tasks.rados.rados.0.smithi175.stdout:372: read oid 34 snap 26 2024-09-15T01:33:40.114 INFO:tasks.rados.rados.0.smithi175.stdout:372: expect deleted 2024-09-15T01:33:40.114 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 7 current snap is 33 2024-09-15T01:33:40.114 INFO:tasks.rados.rados.0.smithi175.stdout:373: seq_num 152 ranges {2818048=147456} 2024-09-15T01:33:40.130 INFO:tasks.rados.rados.0.smithi175.stdout:373: writing smithi17529310-7 from 2818048 to 2965504 tid 1 2024-09-15T01:33:40.130 INFO:tasks.rados.rados.0.smithi175.stdout:374: setattr oid 29 current snap is 33 2024-09-15T01:33:40.130 INFO:tasks.rados.rados.0.smithi175.stdout:363: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-15T01:33:40.137 INFO:tasks.rados.rados.0.smithi175.stdout:364: expect (ObjNum 139 snap 29 seq_num 139) 2024-09-15T01:33:40.147 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:40.146+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:40.157 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:40.157 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:40.157 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:40.157 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:40.157 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:40.166 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:33:40.204 INFO:tasks.rados.rados.0.smithi175.stdout:366: expect (ObjNum 68 snap 3 seq_num 68) 2024-09-15T01:33:40.248 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:40.245+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:40.254 INFO:tasks.rados.rados.0.smithi175.stdout:370: finishing rollback tid 1 to smithi17529310-1 2024-09-15T01:33:40.254 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 1 v 0 (ObjNum 65 snap 0 seq_num 1639060432) dirty dne 2024-09-15T01:33:40.254 INFO:tasks.rados.rados.0.smithi175.stdout:369: finishing copy_from to smithi17529310-22 2024-09-15T01:33:40.254 INFO:tasks.rados.rados.0.smithi175.stdout:369: got expected ENOENT (src dne) 2024-09-15T01:33:40.254 INFO:tasks.rados.rados.0.smithi175.stdout:369: finishing copy_from racing read to smithi17529310-22 2024-09-15T01:33:40.254 INFO:tasks.rados.rados.0.smithi175.stdout:373: finishing write tid 1 to smithi17529310-7 2024-09-15T01:33:40.254 INFO:tasks.rados.rados.0.smithi175.stdout:373: finishing write tid 2 to smithi17529310-7 2024-09-15T01:33:40.254 INFO:tasks.rados.rados.0.smithi175.stdout:371: expect (ObjNum 84 snap 7 seq_num 84) 2024-09-15T01:33:40.267 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:40.267 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:40.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:40.267 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:40.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.267 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:40.267 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:40.267 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.017336120000003, 2024-09-15T01:33:40.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011534543, 2024-09-15T01:33:40.267 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246894+0000", 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0500000000000002e-07 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2585000000000001e-05 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246951+0000", 2024-09-15T01:33:40.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4552999999999999e-05 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246961+0000", 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0516000000000001e-05 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247025+0000", 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3540999999999994e-05 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247039+0000", 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4484000000000001e-05 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247338+0000", 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00029853699999999998 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248324+0000", 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0009866059999999999 2024-09-15T01:33:40.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248375+0000", 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0337000000000003e-05 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248387+0000", 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2081e-05 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258427+0000", 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010040498 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as6 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.016691981000001, 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010349413, 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.273 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247538+0000", 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0130000000000001e-06 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247552+0000", 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3723999999999999e-05 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247588+0000", 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5989999999999999e-05 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247595+0000", 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5789999999999998e-06 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247648+0000", 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2757000000000003e-05 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247654+0000", 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3629999999999999e-06 2024-09-15T01:33:40.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247737+0000", 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2886000000000006e-05 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248326+0000", 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058858900000000004 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248380+0000", 2024-09-15T01:33:40.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3832999999999998e-05 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248401+0000", 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0843e-05 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257886+0000", 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0094858359999999992 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.005160627000002, 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015398044, 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259069+0000", 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2399999999999996e-07 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259095+0000", 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5527000000000001e-05 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259133+0000", 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8211999999999999e-05 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259140+0000", 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9330000000000004e-06 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259189+0000", 2024-09-15T01:33:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8797000000000001e-05 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259200+0000", 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0932e-05 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259425+0000", 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000225505 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259453+0000", 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8152000000000001e-05 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.261606+0000", 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021525870000000001 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.261621+0000", 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4829e-05 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274466+0000", 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012845646 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 44/31 ECSubWrite(tid=39, reqid=client.4477.0:283, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.995836418000003, 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014946894, 2024-09-15T01:33:40.284 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0299999999999998e-07 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268593+0000", 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000199278 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268633+0000", 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0367000000000002e-05 2024-09-15T01:33:40.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268640+0000", 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5100000000000001e-06 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268742+0000", 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010184899999999999 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268752+0000", 2024-09-15T01:33:40.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9299999999999998e-06 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269007+0000", 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000255016 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269035+0000", 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8039999999999999e-05 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.288 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271592+0000", 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025566260000000002 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271610+0000", 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7629999999999999e-05 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283339+0000", 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011729844999999999 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.289 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.6s6 44/38 ECSubWrite(tid=22, reqid=client.4477.0:293, at_version=44'14, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.993052896999998, 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012212746, 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:40.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271177+0000", 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271394+0000", 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021719400000000001 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271421+0000", 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6842000000000002e-05 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271426+0000", 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9010000000000002e-06 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271464+0000", 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7928e-05 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271476+0000", 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1979e-05 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271800+0000", 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032464100000000002 2024-09-15T01:33:40.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271832+0000", 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1501999999999998e-05 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271921+0000", 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9514999999999999e-05 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.294 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271940+0000", 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8436000000000001e-05 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283389+0000", 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011449086000000001 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 55/31 ECSubWrite(tid=160, reqid=client.4477.0:512, at_version=55'53, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:40.295 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 43.188552223000002, 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011287287999999999, 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:40.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075698+0000", 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0384999999999999e-05 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075734+0000", 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6300999999999997e-05 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075743+0000", 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9549999999999998e-06 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075803+0000", 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0304999999999999e-05 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075813+0000", 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9539999999999999e-06 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075916+0000", 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102534 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075925+0000", 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1509999999999993e-06 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075971+0000", 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6276999999999998e-05 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075995+0000", 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3176999999999998e-05 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.086964+0000", 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010969504999999999 2024-09-15T01:33:40.300 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 55/31 ECSubWrite(tid=161, reqid=client.4477.0:513, at_version=55'54, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 43.187915779000001, 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010838802, 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076314+0000", 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076329+0000", 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5505000000000001e-05 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076356+0000", 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6324000000000001e-05 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076362+0000", 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2820000000000002e-06 2024-09-15T01:33:40.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076409+0000", 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7190000000000001e-05 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076419+0000", 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9450000000000005e-06 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076491+0000", 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1680999999999994e-05 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076497+0000", 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3810000000000004e-06 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076958+0000", 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00046042499999999998 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076971+0000", 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2999999999999999e-05 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.087152+0000", 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010181366000000001 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 59/31 ECSubWrite(tid=179, reqid=client.4477.0:552, at_version=59'58, trim_to=0'0, roll_forward_to=56'56))", 2024-09-15T01:33:40.306 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 39.172746510000003, 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010784815, 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:40.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091483+0000", 2024-09-15T01:33:40.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2999999999999999e-07 2024-09-15T01:33:40.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091650+0000", 2024-09-15T01:33:40.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016715 2024-09-15T01:33:40.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091673+0000", 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3434999999999998e-05 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091680+0000", 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7329999999999997e-06 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091721+0000", 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1088999999999999e-05 2024-09-15T01:33:40.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091729+0000", 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9170000000000006e-06 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091940+0000", 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021130000000000001 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.092580+0000", 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00063981200000000002 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093157+0000", 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057673800000000003 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093178+0000", 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0911e-05 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.102267+0000", 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090895000000000004 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 66/31 ECSubWrite(tid=64, reqid=client.4477.0:609, at_version=66'56, trim_to=0'0, roll_forward_to=66'52))", 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:08.087598+0000", 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 32.176631002999997, 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010297796999999999, 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.312 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.087598+0000", 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.087598+0000", 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.087599+0000", 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1100000000000005e-07 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088049+0000", 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045032600000000002 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088087+0000", 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8138e-05 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088095+0000", 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.34e-06 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088153+0000", 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-05 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088165+0000", 2024-09-15T01:33:40.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1386000000000001e-05 2024-09-15T01:33:40.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088485+0000", 2024-09-15T01:33:40.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032002700000000002 2024-09-15T01:33:40.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088493+0000", 2024-09-15T01:33:40.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8550000000000008e-06 2024-09-15T01:33:40.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088596+0000", 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102278 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088613+0000", 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7757e-05 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.097896+0000", 2024-09-15T01:33:40.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0092822389999999994 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 71/31 ECSubWrite(tid=229, reqid=client.4477.0:651, at_version=71'127, trim_to=0'0, roll_forward_to=71'124))", 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.208012+0000", 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.056216598999999, 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.016322817, 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208012+0000", 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208012+0000", 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208013+0000", 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3099999999999997e-07 2024-09-15T01:33:40.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208033+0000", 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9539e-05 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208064+0000", 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1705999999999998e-05 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208073+0000", 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3750000000000003e-06 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208136+0000", 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3736000000000001e-05 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208146+0000", 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9029999999999999e-06 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208264+0000", 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117372 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208272+0000", 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.755e-06 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208300+0000", 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7875999999999999e-05 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208320+0000", 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9791000000000001e-05 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224335+0000", 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.016015033000000001 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.323 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 71/31 ECSubWrite(tid=231, reqid=client.4477.0:652, at_version=71'128, trim_to=0'0, roll_forward_to=71'124))", 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.209071+0000", 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.055157822000002, 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015677593, 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209071+0000", 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209071+0000", 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209072+0000", 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1399999999999997e-07 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209084+0000", 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1917e-05 2024-09-15T01:33:40.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.326 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209402+0000", 2024-09-15T01:33:40.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00031803499999999999 2024-09-15T01:33:40.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.326 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209407+0000", 2024-09-15T01:33:40.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3179999999999997e-06 2024-09-15T01:33:40.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.326 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209454+0000", 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6981000000000003e-05 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209464+0000", 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0037999999999999e-05 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209592+0000", 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012783 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209599+0000", 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9489999999999999e-06 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209779+0000", 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017987 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209797+0000", 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8071e-05 2024-09-15T01:33:40.328 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224749+0000", 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.01495207 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 71/31 ECSubWrite(tid=232, reqid=client.4477.0:652, at_version=0'0, trim_to=0'0, roll_forward_to=71'128))", 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.053748227, 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014733626, 2024-09-15T01:33:40.329 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8999999999999996e-07 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210491+0000", 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7669999999999998e-06 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210530+0000", 2024-09-15T01:33:40.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9039999999999999e-05 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210538+0000", 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2029999999999992e-06 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210601+0000", 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3069000000000002e-05 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210612+0000", 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0372e-05 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210761+0000", 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000149044 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210833+0000", 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2084999999999998e-05 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.333 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210854+0000", 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1424999999999999e-05 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210863+0000", 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1430000000000004e-06 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.225214+0000", 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014350887999999999 2024-09-15T01:33:40.334 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=233, reqid=client.4477.0:653, at_version=0'0, trim_to=0'0, roll_forward_to=71'67))", 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.053734433999999, 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013895739000000001, 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:40.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0599999999999999e-07 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210501+0000", 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2310000000000001e-06 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210562+0000", 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0612000000000002e-05 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210570+0000", 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4109999999999996e-06 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210619+0000", 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9419000000000002e-05 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210625+0000", 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5339999999999999e-06 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210691+0000", 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5905e-05 2024-09-15T01:33:40.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210698+0000", 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0910000000000004e-06 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210831+0000", 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000133065 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210844+0000", 2024-09-15T01:33:40.339 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3349e-05 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224390+0000", 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013545916 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=234, reqid=client.4477.0:654, at_version=71'68, trim_to=0'0, roll_forward_to=71'67))", 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:40.340 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.05258401, 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013353726, 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2900000000000004e-07 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212326+0000", 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00068024600000000002 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212354+0000", 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8555999999999999e-05 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212360+0000", 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.755e-06 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212397+0000", 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7086999999999999e-05 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212404+0000", 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7190000000000004e-06 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212510+0000", 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010600800000000001 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212515+0000", 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3159999999999996e-06 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213020+0000", 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000504854 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213032+0000", 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2048999999999999e-05 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224999+0000", 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011966607000000001 2024-09-15T01:33:40.345 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=235, reqid=client.4477.0:654, at_version=0'0, trim_to=0'0, roll_forward_to=71'68))", 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.050677178000001, 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011535082, 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.346 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4799999999999998e-07 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.347 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213560+0000", 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213582+0000", 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2524e-05 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213588+0000", 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9009999999999999e-06 2024-09-15T01:33:40.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213624+0000", 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5725000000000001e-05 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213629+0000", 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0679999999999996e-06 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213684+0000", 2024-09-15T01:33:40.349 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4722999999999998e-05 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213690+0000", 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4489999999999996e-06 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.215563+0000", 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018726680000000001 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.215576+0000", 2024-09-15T01:33:40.350 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3021e-05 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.225087+0000", 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0095110449999999992 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 85/31 ECSubWrite(tid=351, reqid=osd.11.0:351, at_version=85'111, trim_to=0'0, roll_forward_to=85'107))", 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:27.141013+0000", 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 13.123216383000001, 2024-09-15T01:33:40.351 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.109127261, 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.141013+0000", 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.141013+0000", 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.141013+0000", 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.06e-07 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.141037+0000", 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4209000000000001e-05 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.353 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249578+0000", 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.10854032600000001 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249589+0000", 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1396000000000001e-05 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249676+0000", 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6737999999999995e-05 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.354 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249682+0000", 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0859999999999998e-06 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249864+0000", 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018231000000000001 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249872+0000", 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9370000000000004e-06 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.355 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249905+0000", 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2852000000000003e-05 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249918+0000", 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2972e-05 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.250140+0000", 2024-09-15T01:33:40.356 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022172899999999999 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 88/31 ECSubWrite(tid=377, reqid=client.4477.0:877, at_version=88'202, trim_to=0'0, roll_forward_to=87'198))", 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:30.160526+0000", 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 10.103703399, 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015647682999999999, 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.357 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160526+0000", 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160526+0000", 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160526+0000", 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4099999999999997e-07 2024-09-15T01:33:40.358 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160546+0000", 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9592000000000001e-05 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160582+0000", 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5706e-05 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160588+0000", 2024-09-15T01:33:40.359 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7399999999999998e-06 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160647+0000", 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8125999999999999e-05 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160655+0000", 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4470000000000006e-06 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160917+0000", 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00026217299999999999 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160924+0000", 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2609999999999996e-06 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163192+0000", 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022676559999999998 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163210+0000", 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7781000000000001e-05 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.176173+0000", 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.01296336 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.362 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 88/31 ECSubWrite(tid=379, reqid=client.4477.0:878, at_version=88'203, trim_to=0'0, roll_forward_to=87'198))", 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:30.161562+0000", 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 10.102666709999999, 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.016881072, 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161562+0000", 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.363 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161562+0000", 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161563+0000", 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1399999999999996e-07 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161579+0000", 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.641e-05 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161619+0000", 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9190999999999998e-05 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161626+0000", 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6109999999999996e-06 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161673+0000", 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6524999999999999e-05 2024-09-15T01:33:40.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161682+0000", 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5799999999999998e-06 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161833+0000", 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015063199999999999 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163196+0000", 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0013626339999999999 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163354+0000", 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015873699999999999 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163372+0000", 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7430000000000001e-05 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.178443+0000", 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015071508000000001 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 88/31 ECSubWrite(tid=381, reqid=client.4477.0:879, at_version=88'204, trim_to=0'0, roll_forward_to=88'199))", 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:30.162496+0000", 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 10.101732568999999, 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.01606695, 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.368 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162496+0000", 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162496+0000", 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162497+0000", 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8599999999999997e-07 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162517+0000", 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9426e-05 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162545+0000", 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8138999999999999e-05 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162551+0000", 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0970000000000002e-06 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162598+0000", 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7187000000000001e-05 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162611+0000", 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2505e-05 2024-09-15T01:33:40.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162769+0000", 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015802899999999999 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163200+0000", 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043180900000000001 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163359+0000", 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015842699999999999 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163400+0000", 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1702000000000002e-05 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.178563+0000", 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015162843000000001 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.374 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 88/31 ECSubWrite(tid=383, reqid=client.4477.0:879, at_version=0'0, trim_to=0'0, roll_forward_to=88'204))", 2024-09-15T01:33:40.374 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:30.164585+0000", 2024-09-15T01:33:40.374 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 10.099643491, 2024-09-15T01:33:40.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014151115000000001, 2024-09-15T01:33:40.374 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.374 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.374 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164585+0000", 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164585+0000", 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164586+0000", 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3499999999999995e-07 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164598+0000", 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1935e-05 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164624+0000", 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5931e-05 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164630+0000", 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6099999999999997e-06 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164685+0000", 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5337999999999998e-05 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164691+0000", 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4999999999999996e-06 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164830+0000", 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013819900000000001 2024-09-15T01:33:40.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164840+0000", 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9820000000000003e-06 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.166597+0000", 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001757273 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.166633+0000", 2024-09-15T01:33:40.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6386e-05 2024-09-15T01:33:40.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.178737+0000", 2024-09-15T01:33:40.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012103226 2024-09-15T01:33:40.379 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.379 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.379 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.379 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.379 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.379 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:40.381 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-15T01:33:40.381 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:40.379+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:40.479 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:40.479 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:40.479 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:40.481 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:40.479+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:40.487 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-15T01:33:40.539 INFO:tasks.rados.rados.0.smithi175.stdout:373: finishing write tid 3 to smithi17529310-7 2024-09-15T01:33:40.540 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 7 v 99 (ObjNum 152 snap 33 seq_num 152) dirty exists 2024-09-15T01:33:40.540 INFO:tasks.rados.rados.0.smithi175.stdout:373: left oid 7 (ObjNum 152 snap 33 seq_num 152) 2024-09-15T01:33:40.540 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 29 v 233 (ObjNum 123 snap 20 seq_num 123) dirty exists 2024-09-15T01:33:40.540 INFO:tasks.rados.rados.0.smithi175.stdout:362: done (9 left) 2024-09-15T01:33:40.540 INFO:tasks.rados.rados.0.smithi175.stdout:363: done (8 left) 2024-09-15T01:33:40.540 INFO:tasks.rados.rados.0.smithi175.stdout:364: done (7 left) 2024-09-15T01:33:40.540 INFO:tasks.rados.rados.0.smithi175.stdout:366: done (6 left) 2024-09-15T01:33:40.540 INFO:tasks.rados.rados.0.smithi175.stdout:369: done (5 left) 2024-09-15T01:33:40.540 INFO:tasks.rados.rados.0.smithi175.stdout:370: done (4 left) 2024-09-15T01:33:40.540 INFO:tasks.rados.rados.0.smithi175.stdout:371: done (3 left) 2024-09-15T01:33:40.540 INFO:tasks.rados.rados.0.smithi175.stdout:372: done (2 left) 2024-09-15T01:33:40.540 INFO:tasks.rados.rados.0.smithi175.stdout:373: done (1 left) 2024-09-15T01:33:40.541 INFO:tasks.rados.rados.0.smithi175.stdout:374: done (0 left) 2024-09-15T01:33:40.541 INFO:tasks.rados.rados.0.smithi175.stdout:375: setattr oid 46 current snap is 33 2024-09-15T01:33:40.541 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 3 current snap is 33 2024-09-15T01:33:40.541 INFO:tasks.rados.rados.0.smithi175.stdout:376: seq_num 153 ranges {3620864=163840} 2024-09-15T01:33:40.561 INFO:tasks.rados.rados.0.smithi175.stdout:376: writing smithi17529310-3 from 3620864 to 3784704 tid 1 2024-09-15T01:33:40.561 INFO:tasks.rados.rados.0.smithi175.stdout:377: read oid 37 snap -1 2024-09-15T01:33:40.561 INFO:tasks.rados.rados.0.smithi175.stdout:377: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-15T01:33:40.561 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 5 current snap is 33 2024-09-15T01:33:40.561 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 46 v 133 (ObjNum 135 snap 24 seq_num 135) dirty exists 2024-09-15T01:33:40.561 INFO:tasks.rados.rados.0.smithi175.stdout:378: seq_num 154 ranges {1687552=180224} 2024-09-15T01:33:40.571 INFO:tasks.rados.rados.0.smithi175.stdout:378: writing smithi17529310-5 from 1687552 to 1867776 tid 1 2024-09-15T01:33:40.571 INFO:tasks.rados.rados.0.smithi175.stdout:375: done (3 left) 2024-09-15T01:33:40.571 INFO:tasks.rados.rados.0.smithi175.stdout:379: rmattr oid 24 current snap is 33 2024-09-15T01:33:40.571 INFO:tasks.rados.rados.0.smithi175.stdout:379: done (3 left) 2024-09-15T01:33:40.571 INFO:tasks.rados.rados.0.smithi175.stdout:380: snap_create 2024-09-15T01:33:40.571 INFO:tasks.rados.rados.0.smithi175.stdout:376: finishing write tid 1 to smithi17529310-3 2024-09-15T01:33:40.571 INFO:tasks.rados.rados.0.smithi175.stdout:376: finishing write tid 2 to smithi17529310-3 2024-09-15T01:33:40.571 INFO:tasks.rados.rados.0.smithi175.stdout:377: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-15T01:33:40.581 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:40.580+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:40.587 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:40.587 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:40.587 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:40.587 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:40.587 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:40.590 INFO:tasks.rados.rados.0.smithi175.stdout:378: finishing write tid 1 to smithi17529310-5 2024-09-15T01:33:40.590 INFO:tasks.rados.rados.0.smithi175.stdout:378: finishing write tid 2 to smithi17529310-5 2024-09-15T01:33:40.591 INFO:tasks.rados.rados.0.smithi175.stdout:376: finishing write tid 3 to smithi17529310-3 2024-09-15T01:33:40.591 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 3 v 134 (ObjNum 153 snap 33 seq_num 153) dirty exists 2024-09-15T01:33:40.591 INFO:tasks.rados.rados.0.smithi175.stdout:376: left oid 3 (ObjNum 153 snap 33 seq_num 153) 2024-09-15T01:33:40.591 INFO:tasks.rados.rados.0.smithi175.stdout:378: finishing write tid 3 to smithi17529310-5 2024-09-15T01:33:40.591 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 5 v 32 (ObjNum 154 snap 33 seq_num 154) dirty exists 2024-09-15T01:33:40.591 INFO:tasks.rados.rados.0.smithi175.stdout:378: left oid 5 (ObjNum 154 snap 33 seq_num 154) 2024-09-15T01:33:40.596 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-15T01:33:40.672 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 9] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-15T01:33:40.672 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:33:40.673 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2024-09-15T01:33:40.673 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2024-09-15T01:33:40.718 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:40.716+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:40.733 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:40.734 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:40.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:40.734 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:40.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.734 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:74 3.0s0 3:0187269c:::smithi17529310-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:40.734 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:40.734 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 64.520483855999998, 2024-09-15T01:33:40.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.071208814999999995, 2024-09-15T01:33:40.734 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.735 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:40.735 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:40.735 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:40.735 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:40.735 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 74 2024-09-15T01:33:40.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.735 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:40.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:40.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:40.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1200000000000003e-07 2024-09-15T01:33:40.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210620+0000", 2024-09-15T01:33:40.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.278e-06 2024-09-15T01:33:40.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210623+0000", 2024-09-15T01:33:40.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5779999999999999e-06 2024-09-15T01:33:40.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210626+0000", 2024-09-15T01:33:40.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.295e-06 2024-09-15T01:33:40.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211745+0000", 2024-09-15T01:33:40.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011191770000000001 2024-09-15T01:33:40.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:40.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211880+0000", 2024-09-15T01:33:40.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013497499999999999 2024-09-15T01:33:40.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212353+0000", 2024-09-15T01:33:40.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000473616 2024-09-15T01:33:40.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212441+0000", 2024-09-15T01:33:40.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7268999999999994e-05 2024-09-15T01:33:40.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212454+0000", 2024-09-15T01:33:40.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3529999999999999e-05 2024-09-15T01:33:40.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212483+0000", 2024-09-15T01:33:40.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9224e-05 2024-09-15T01:33:40.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212755+0000", 2024-09-15T01:33:40.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00027191599999999998 2024-09-15T01:33:40.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:33:40.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.281770+0000", 2024-09-15T01:33:40.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.069014438999999997 2024-09-15T01:33:40.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.281826+0000", 2024-09-15T01:33:40.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6005999999999999e-05 2024-09-15T01:33:40.742 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.742 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.742 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.742 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:75 3.0s0 3:0187269c:::smithi17529310-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:40.742 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:40.742 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 64.520451616000003, 2024-09-15T01:33:40.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.072451382999999994, 2024-09-15T01:33:40.743 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.743 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:33:40.743 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:40.743 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:40.743 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:40.743 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 75 2024-09-15T01:33:40.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.743 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:40.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:40.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:40.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4700000000000002e-07 2024-09-15T01:33:40.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210652+0000", 2024-09-15T01:33:40.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9160000000000001e-06 2024-09-15T01:33:40.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210657+0000", 2024-09-15T01:33:40.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0529999999999997e-06 2024-09-15T01:33:40.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210660+0000", 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4339999999999999e-06 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212500+0000", 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001839732 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212584+0000", 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4759999999999995e-05 2024-09-15T01:33:40.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.282036+0000", 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.069451322999999995 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.282131+0000", 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.577e-05 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.283100+0000", 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00096904800000000002 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds2 34/31 ECSubWrite(tid=16, reqid=client.4477.0:116, at_version=34'10, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 64.471477167000003, 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.24562283800000001, 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.748 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:40.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9599999999999999e-07 2024-09-15T01:33:40.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259634+0000", 2024-09-15T01:33:40.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0412e-05 2024-09-15T01:33:40.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.504838+0000", 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.245203848 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.504852+0000", 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3346999999999999e-05 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505000+0000", 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000148209 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505006+0000", 2024-09-15T01:33:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6309999999999997e-06 2024-09-15T01:33:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505068+0000", 2024-09-15T01:33:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1962000000000001e-05 2024-09-15T01:33:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505074+0000", 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0579999999999997e-06 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505103+0000", 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9828e-05 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505116+0000", 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2261e-05 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505246+0000", 2024-09-15T01:33:40.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000130786 2024-09-15T01:33:40.754 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.754 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.754 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.754 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds2 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:40.754 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:40.755 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.484184519999999, 2024-09-15T01:33:40.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033387655000000002, 2024-09-15T01:33:40.755 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.755 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.755 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:40.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:40.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246917+0000", 2024-09-15T01:33:40.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.61e-07 2024-09-15T01:33:40.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246937+0000", 2024-09-15T01:33:40.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9741999999999998e-05 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246964+0000", 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7013999999999999e-05 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246970+0000", 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8250000000000001e-06 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247020+0000", 2024-09-15T01:33:40.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0875999999999999e-05 2024-09-15T01:33:40.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247032+0000", 2024-09-15T01:33:40.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1280999999999999e-05 2024-09-15T01:33:40.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249500+0000", 2024-09-15T01:33:40.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0024682720000000001 2024-09-15T01:33:40.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251106+0000", 2024-09-15T01:33:40.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001606489 2024-09-15T01:33:40.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251175+0000", 2024-09-15T01:33:40.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8926999999999999e-05 2024-09-15T01:33:40.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251258+0000", 2024-09-15T01:33:40.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2441000000000006e-05 2024-09-15T01:33:40.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280304+0000", 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029046026999999999 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs2 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247251+0000", 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.483849894000002, 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033110460000000001, 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.761 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247251+0000", 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247251+0000", 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247252+0000", 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9899999999999999e-07 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247274+0000", 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2160000000000001e-05 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247314+0000", 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0438000000000001e-05 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247321+0000", 2024-09-15T01:33:40.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.187e-06 2024-09-15T01:33:40.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249348+0000", 2024-09-15T01:33:40.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.002026524 2024-09-15T01:33:40.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249356+0000", 2024-09-15T01:33:40.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7319999999999998e-06 2024-09-15T01:33:40.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249538+0000", 2024-09-15T01:33:40.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000182239 2024-09-15T01:33:40.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251107+0000", 2024-09-15T01:33:40.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001569094 2024-09-15T01:33:40.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251176+0000", 2024-09-15T01:33:40.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8707999999999993e-05 2024-09-15T01:33:40.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251269+0000", 2024-09-15T01:33:40.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3246999999999997e-05 2024-09-15T01:33:40.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280361+0000", 2024-09-15T01:33:40.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029092331999999999 2024-09-15T01:33:40.768 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.768 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.768 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.768 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s5 44/31 ECSubWrite(tid=28, reqid=osd.8.0:28, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:40.768 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.250215+0000", 2024-09-15T01:33:40.768 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.480885723999997, 2024-09-15T01:33:40.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033811606000000001, 2024-09-15T01:33:40.769 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.769 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.769 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250215+0000", 2024-09-15T01:33:40.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250215+0000", 2024-09-15T01:33:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250216+0000", 2024-09-15T01:33:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3200000000000003e-07 2024-09-15T01:33:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250240+0000", 2024-09-15T01:33:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3849000000000002e-05 2024-09-15T01:33:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250277+0000", 2024-09-15T01:33:40.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7469999999999999e-05 2024-09-15T01:33:40.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250284+0000", 2024-09-15T01:33:40.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0979999999999996e-06 2024-09-15T01:33:40.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250325+0000", 2024-09-15T01:33:40.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0184e-05 2024-09-15T01:33:40.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250331+0000", 2024-09-15T01:33:40.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5060000000000001e-06 2024-09-15T01:33:40.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254385+0000", 2024-09-15T01:33:40.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0040539560000000001 2024-09-15T01:33:40.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254397+0000", 2024-09-15T01:33:40.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2502e-05 2024-09-15T01:33:40.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254432+0000", 2024-09-15T01:33:40.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4631000000000001e-05 2024-09-15T01:33:40.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254987+0000", 2024-09-15T01:33:40.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000555011 2024-09-15T01:33:40.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.284027+0000", 2024-09-15T01:33:40.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029039466999999999 2024-09-15T01:33:40.775 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.775 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.775 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.775 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s5 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:40.775 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.251962+0000", 2024-09-15T01:33:40.775 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.479138409000001, 2024-09-15T01:33:40.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034678124999999997, 2024-09-15T01:33:40.775 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.776 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.776 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251962+0000", 2024-09-15T01:33:40.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251962+0000", 2024-09-15T01:33:40.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251963+0000", 2024-09-15T01:33:40.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4799999999999998e-07 2024-09-15T01:33:40.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251986+0000", 2024-09-15T01:33:40.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3221e-05 2024-09-15T01:33:40.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252099+0000", 2024-09-15T01:33:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112687 2024-09-15T01:33:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252107+0000", 2024-09-15T01:33:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0399999999999993e-06 2024-09-15T01:33:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254434+0000", 2024-09-15T01:33:40.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0023271300000000002 2024-09-15T01:33:40.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254447+0000", 2024-09-15T01:33:40.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2607e-05 2024-09-15T01:33:40.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254678+0000", 2024-09-15T01:33:40.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000230869 2024-09-15T01:33:40.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254847+0000", 2024-09-15T01:33:40.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016962300000000001 2024-09-15T01:33:40.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254891+0000", 2024-09-15T01:33:40.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3441e-05 2024-09-15T01:33:40.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.255014+0000", 2024-09-15T01:33:40.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012353200000000001 2024-09-15T01:33:40.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286640+0000", 2024-09-15T01:33:40.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031626327000000003 2024-09-15T01:33:40.781 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.781 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.782 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.782 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs2 44/31 ECSubWrite(tid=91, reqid=osd.11.0:87, at_version=0'0, trim_to=0'0, roll_forward_to=44'44))", 2024-09-15T01:33:40.782 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267434+0000", 2024-09-15T01:33:40.782 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.463666748999998, 2024-09-15T01:33:40.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039301718999999999, 2024-09-15T01:33:40.782 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.782 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.782 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267434+0000", 2024-09-15T01:33:40.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267434+0000", 2024-09-15T01:33:40.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267435+0000", 2024-09-15T01:33:40.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.92e-07 2024-09-15T01:33:40.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267445+0000", 2024-09-15T01:33:40.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8949999999999993e-06 2024-09-15T01:33:40.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267477+0000", 2024-09-15T01:33:40.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2768000000000002e-05 2024-09-15T01:33:40.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267485+0000", 2024-09-15T01:33:40.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4499999999999998e-06 2024-09-15T01:33:40.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281136+0000", 2024-09-15T01:33:40.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013651389999999999 2024-09-15T01:33:40.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281144+0000", 2024-09-15T01:33:40.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9549999999999992e-06 2024-09-15T01:33:40.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281264+0000", 2024-09-15T01:33:40.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012005799999999999 2024-09-15T01:33:40.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282867+0000", 2024-09-15T01:33:40.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00160271 2024-09-15T01:33:40.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282917+0000", 2024-09-15T01:33:40.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9919000000000001e-05 2024-09-15T01:33:40.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282948+0000", 2024-09-15T01:33:40.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1053000000000002e-05 2024-09-15T01:33:40.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306736+0000", 2024-09-15T01:33:40.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023787729000000001 2024-09-15T01:33:40.788 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.788 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.788 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.788 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s6 44/38 ECSubWrite(tid=60, reqid=client.4477.0:306, at_version=44'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:40.789 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270872+0000", 2024-09-15T01:33:40.789 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.460228686999997, 2024-09-15T01:33:40.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039161770999999998, 2024-09-15T01:33:40.789 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.789 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.789 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270872+0000", 2024-09-15T01:33:40.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270872+0000", 2024-09-15T01:33:40.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270873+0000", 2024-09-15T01:33:40.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0219999999999999e-06 2024-09-15T01:33:40.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270902+0000", 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9079000000000001e-05 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270938+0000", 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5849999999999997e-05 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270958+0000", 2024-09-15T01:33:40.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9762000000000002e-05 2024-09-15T01:33:40.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280987+0000", 2024-09-15T01:33:40.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010029733000000001 2024-09-15T01:33:40.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280996+0000", 2024-09-15T01:33:40.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0280000000000006e-06 2024-09-15T01:33:40.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281214+0000", 2024-09-15T01:33:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021798100000000001 2024-09-15T01:33:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282866+0000", 2024-09-15T01:33:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001651292 2024-09-15T01:33:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282916+0000", 2024-09-15T01:33:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0145000000000002e-05 2024-09-15T01:33:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282941+0000", 2024-09-15T01:33:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5066e-05 2024-09-15T01:33:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.310034+0000", 2024-09-15T01:33:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027092813 2024-09-15T01:33:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.795 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.795 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.795 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s4 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:40.795 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270969+0000", 2024-09-15T01:33:40.795 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.460131517999997, 2024-09-15T01:33:40.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034685404000000003, 2024-09-15T01:33:40.795 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.795 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.795 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270969+0000", 2024-09-15T01:33:40.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270969+0000", 2024-09-15T01:33:40.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270970+0000", 2024-09-15T01:33:40.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6199999999999998e-07 2024-09-15T01:33:40.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270986+0000", 2024-09-15T01:33:40.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6622000000000001e-05 2024-09-15T01:33:40.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271019+0000", 2024-09-15T01:33:40.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3077000000000002e-05 2024-09-15T01:33:40.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271025+0000", 2024-09-15T01:33:40.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8699999999999997e-06 2024-09-15T01:33:40.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274545+0000", 2024-09-15T01:33:40.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0035193350000000002 2024-09-15T01:33:40.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274553+0000", 2024-09-15T01:33:40.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2810000000000008e-06 2024-09-15T01:33:40.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274633+0000", 2024-09-15T01:33:40.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0358000000000005e-05 2024-09-15T01:33:40.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274827+0000", 2024-09-15T01:33:40.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019425000000000001 2024-09-15T01:33:40.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.276977+0000", 2024-09-15T01:33:40.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021494420000000001 2024-09-15T01:33:40.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.276994+0000", 2024-09-15T01:33:40.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6733000000000001e-05 2024-09-15T01:33:40.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.305655+0000", 2024-09-15T01:33:40.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028660973999999999 2024-09-15T01:33:40.801 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.801 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.801 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.802 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s4 44/31 ECSubWrite(tid=34, reqid=client.4477.0:296, at_version=44'25, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:40.802 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:40.802 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.459618204999998, 2024-09-15T01:33:40.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034642487, 2024-09-15T01:33:40.802 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.802 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.802 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:40.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:40.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:40.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2600000000000002e-07 2024-09-15T01:33:40.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271500+0000", 2024-09-15T01:33:40.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6745999999999999e-05 2024-09-15T01:33:40.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271570+0000", 2024-09-15T01:33:40.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0242999999999998e-05 2024-09-15T01:33:40.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271578+0000", 2024-09-15T01:33:40.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0870000000000007e-06 2024-09-15T01:33:40.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274674+0000", 2024-09-15T01:33:40.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030959379999999999 2024-09-15T01:33:40.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274682+0000", 2024-09-15T01:33:40.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9570000000000002e-06 2024-09-15T01:33:40.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274766+0000", 2024-09-15T01:33:40.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3630999999999999e-05 2024-09-15T01:33:40.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274828+0000", 2024-09-15T01:33:40.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2609999999999999e-05 2024-09-15T01:33:40.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.276978+0000", 2024-09-15T01:33:40.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021496520000000002 2024-09-15T01:33:40.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277024+0000", 2024-09-15T01:33:40.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5723999999999998e-05 2024-09-15T01:33:40.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306125+0000", 2024-09-15T01:33:40.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029101373 2024-09-15T01:33:40.808 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.809 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.809 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.809 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s6 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:40.809 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.272445+0000", 2024-09-15T01:33:40.809 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.458655387999997, 2024-09-15T01:33:40.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041201330000000001, 2024-09-15T01:33:40.809 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.809 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.809 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272445+0000", 2024-09-15T01:33:40.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272445+0000", 2024-09-15T01:33:40.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272446+0000", 2024-09-15T01:33:40.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8299999999999995e-07 2024-09-15T01:33:40.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272467+0000", 2024-09-15T01:33:40.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0319999999999999e-05 2024-09-15T01:33:40.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272505+0000", 2024-09-15T01:33:40.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8862999999999997e-05 2024-09-15T01:33:40.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272513+0000", 2024-09-15T01:33:40.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.362e-06 2024-09-15T01:33:40.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.813 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.813 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281243+0000", 2024-09-15T01:33:40.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0087304259999999995 2024-09-15T01:33:40.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.813 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.813 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281250+0000", 2024-09-15T01:33:40.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9399999999999996e-06 2024-09-15T01:33:40.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.813 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281315+0000", 2024-09-15T01:33:40.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4664000000000003e-05 2024-09-15T01:33:40.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282868+0000", 2024-09-15T01:33:40.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001552793 2024-09-15T01:33:40.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282917+0000", 2024-09-15T01:33:40.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9567999999999997e-05 2024-09-15T01:33:40.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282975+0000", 2024-09-15T01:33:40.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7617000000000001e-05 2024-09-15T01:33:40.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.313647+0000", 2024-09-15T01:33:40.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030671894000000002 2024-09-15T01:33:40.815 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.816 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.816 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.816 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s6 44/38 ECSubWrite(tid=62, reqid=client.4477.0:308, at_version=44'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:40.816 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273497+0000", 2024-09-15T01:33:40.816 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 54.457603519000003, 2024-09-15T01:33:40.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.040229292, 2024-09-15T01:33:40.816 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.817 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.817 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.817 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.817 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273497+0000", 2024-09-15T01:33:40.817 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.817 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.817 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.817 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273497+0000", 2024-09-15T01:33:40.817 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.817 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273498+0000", 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2799999999999996e-07 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273514+0000", 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5942000000000001e-05 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273538+0000", 2024-09-15T01:33:40.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4705000000000001e-05 2024-09-15T01:33:40.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273543+0000", 2024-09-15T01:33:40.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9649999999999999e-06 2024-09-15T01:33:40.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281338+0000", 2024-09-15T01:33:40.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0077948050000000001 2024-09-15T01:33:40.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281343+0000", 2024-09-15T01:33:40.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6009999999999997e-06 2024-09-15T01:33:40.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281391+0000", 2024-09-15T01:33:40.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7768000000000001e-05 2024-09-15T01:33:40.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282868+0000", 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001477587 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282917+0000", 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9168000000000001e-05 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282987+0000", 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9294000000000004e-05 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.313726+0000", 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030739829 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=100, last_complete=54'36, committed=1, applied=1))", 2024-09-15T01:33:40.822 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 44.634691848000003, 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.063605834999999999, 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:40.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:40.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.01e-07 2024-09-15T01:33:40.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096412+0000", 2024-09-15T01:33:40.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.836e-06 2024-09-15T01:33:40.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.159953+0000", 2024-09-15T01:33:40.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.063540907999999993 2024-09-15T01:33:40.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.159956+0000", 2024-09-15T01:33:40.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0489999999999999e-06 2024-09-15T01:33:40.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.159987+0000", 2024-09-15T01:33:40.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0725999999999997e-05 2024-09-15T01:33:40.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.160015+0000", 2024-09-15T01:33:40.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7915e-05 2024-09-15T01:33:40.826 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.826 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.826 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.827 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:485 3.9s0 3:98b2f2fd:::smithi17529310-14:head [setxattr _header (57) in=7b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:40.827 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:40.827 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 44.566728157, 2024-09-15T01:33:40.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40475048200000002, 2024-09-15T01:33:40.827 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.827 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:40.827 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:40.827 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:40.827 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:40.827 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 485 2024-09-15T01:33:40.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.828 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.828 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.828 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:40.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.828 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.828 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:40.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:40.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6199999999999998e-07 2024-09-15T01:33:40.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164375+0000", 2024-09-15T01:33:40.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0650000000000001e-06 2024-09-15T01:33:40.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164380+0000", 2024-09-15T01:33:40.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9810000000000003e-06 2024-09-15T01:33:40.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164383+0000", 2024-09-15T01:33:40.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.241e-06 2024-09-15T01:33:40.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165443+0000", 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010601409999999999 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165554+0000", 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011091600000000001 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165888+0000", 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000334125 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:40.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165960+0000", 2024-09-15T01:33:40.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1601999999999996e-05 2024-09-15T01:33:40.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:40.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165971+0000", 2024-09-15T01:33:40.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0903999999999999e-05 2024-09-15T01:33:40.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:40.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166005+0000", 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3525999999999997e-05 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166156+0000", 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000151077 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569047+0000", 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.402891102 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.834 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569123+0000", 2024-09-15T01:33:40.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6340000000000004e-05 2024-09-15T01:33:40.834 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.834 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.834 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.834 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:486 3.9s0 3:98b2f2fd:::smithi17529310-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:40.834 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:40.834 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 44.566702106000001, 2024-09-15T01:33:40.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40586654100000003, 2024-09-15T01:33:40.835 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.835 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:33:40.835 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:40.835 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:40.835 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:40.835 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 486 2024-09-15T01:33:40.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.835 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:40.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2800000000000003e-07 2024-09-15T01:33:40.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164402+0000", 2024-09-15T01:33:40.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3440000000000001e-06 2024-09-15T01:33:40.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164409+0000", 2024-09-15T01:33:40.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6839999999999999e-06 2024-09-15T01:33:40.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164417+0000", 2024-09-15T01:33:40.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9140000000000008e-06 2024-09-15T01:33:40.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166021+0000", 2024-09-15T01:33:40.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016044189999999999 2024-09-15T01:33:40.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:40.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166112+0000", 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0402000000000006e-05 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569421+0000", 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40330946099999998 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569534+0000", 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011237499999999999 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.570265+0000", 2024-09-15T01:33:40.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00073161400000000001 2024-09-15T01:33:40.840 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.840 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.840 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.840 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=106, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:40.840 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:40.840 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 44.564903029, 2024-09-15T01:33:40.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.10267822, 2024-09-15T01:33:40.841 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.841 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.841 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:40.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:40.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:40.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0399999999999996e-07 2024-09-15T01:33:40.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166203+0000", 2024-09-15T01:33:40.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2889999999999999e-06 2024-09-15T01:33:40.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268821+0000", 2024-09-15T01:33:40.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.102618771 2024-09-15T01:33:40.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268827+0000", 2024-09-15T01:33:40.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1140000000000002e-06 2024-09-15T01:33:40.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268865+0000", 2024-09-15T01:33:40.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7492e-05 2024-09-15T01:33:40.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268876+0000", 2024-09-15T01:33:40.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.095e-05 2024-09-15T01:33:40.844 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.844 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.844 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.845 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=106, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:40.845 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.166241+0000", 2024-09-15T01:33:40.845 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 44.564859247999998, 2024-09-15T01:33:40.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.075762497999999998, 2024-09-15T01:33:40.845 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.845 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.845 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.845 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166241+0000", 2024-09-15T01:33:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166241+0000", 2024-09-15T01:33:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166242+0000", 2024-09-15T01:33:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6400000000000003e-07 2024-09-15T01:33:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166244+0000", 2024-09-15T01:33:40.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3309999999999998e-06 2024-09-15T01:33:40.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.241879+0000", 2024-09-15T01:33:40.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.075634921999999993 2024-09-15T01:33:40.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.241898+0000", 2024-09-15T01:33:40.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9082000000000001e-05 2024-09-15T01:33:40.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.241991+0000", 2024-09-15T01:33:40.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2833999999999999e-05 2024-09-15T01:33:40.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.242004+0000", 2024-09-15T01:33:40.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2865e-05 2024-09-15T01:33:40.849 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.849 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.849 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.849 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=106, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:40.849 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.166248+0000", 2024-09-15T01:33:40.849 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 44.564852696000003, 2024-09-15T01:33:40.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40313786299999999, 2024-09-15T01:33:40.849 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166248+0000", 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166248+0000", 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166249+0000", 2024-09-15T01:33:40.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6700000000000003e-07 2024-09-15T01:33:40.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166251+0000", 2024-09-15T01:33:40.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.934e-06 2024-09-15T01:33:40.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.568876+0000", 2024-09-15T01:33:40.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.402625446 2024-09-15T01:33:40.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.568887+0000", 2024-09-15T01:33:40.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0553e-05 2024-09-15T01:33:40.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.568964+0000", 2024-09-15T01:33:40.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7303000000000002e-05 2024-09-15T01:33:40.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569386+0000", 2024-09-15T01:33:40.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00042205999999999998 2024-09-15T01:33:40.853 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.854 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.854 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.854 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 81/31 ECSubWriteReply(tid=148, last_complete=81'55, committed=1, applied=1))", 2024-09-15T01:33:40.855 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:23.088718+0000", 2024-09-15T01:33:40.855 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 17.642382639000001, 2024-09-15T01:33:40.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.42277780999999998, 2024-09-15T01:33:40.855 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:40.855 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:40.855 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:40.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:40.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.088718+0000", 2024-09-15T01:33:40.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:40.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.088718+0000", 2024-09-15T01:33:40.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:40.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.088719+0000", 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1999999999999998e-07 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.088725+0000", 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7209999999999999e-06 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.511335+0000", 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.42261066600000002 2024-09-15T01:33:40.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:40.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.511351+0000", 2024-09-15T01:33:40.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5153e-05 2024-09-15T01:33:40.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:40.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.511467+0000", 2024-09-15T01:33:40.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011659600000000001 2024-09-15T01:33:40.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:40.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:40.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:40.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.511496+0000", 2024-09-15T01:33:40.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8753999999999999e-05 2024-09-15T01:33:40.859 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.859 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.859 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.859 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:40.859 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:40.859 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:40.861 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:33:40.861 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:40.858+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:40.959 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:40.959 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:40.959 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:40.959 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:40.961 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:40.958+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:40.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:33:41.062 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:41.061+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:41.073 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-15T01:33:41.071+0000 7f79b23f6640 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-15T01:33:41.076 INFO:teuthology.orchestra.run.smithi042.stderr:osd set-full-ratio 0.001 2024-09-15T01:33:41.079 INFO:tasks.rados.rados.0.smithi175.stdout:376: done (3 left) 2024-09-15T01:33:41.079 INFO:tasks.rados.rados.0.smithi175.stdout:377: done (2 left) 2024-09-15T01:33:41.079 INFO:tasks.rados.rados.0.smithi175.stdout:378: done (1 left) 2024-09-15T01:33:41.079 INFO:tasks.rados.rados.0.smithi175.stdout:380: done (0 left) 2024-09-15T01:33:41.079 INFO:tasks.rados.rados.0.smithi175.stdout:381: read oid 4 snap -1 2024-09-15T01:33:41.079 INFO:tasks.rados.rados.0.smithi175.stdout:381: expect (ObjNum 124 snap 20 seq_num 124) 2024-09-15T01:33:41.079 INFO:tasks.rados.rados.0.smithi175.stdout:382: snap_remove snap 25 2024-09-15T01:33:41.110 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:41.110 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:41.110 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:41.110 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:41.110 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:41.118 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:33:41.163 INFO:tasks.ceph.osd.3.smithi175.stderr:2024-09-15T01:33:41.161+0000 7f399a7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 28825) UID: 0 2024-09-15T01:33:41.218 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:41.218 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:41.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:41.218 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:41.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.218 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:192 3.as0 3:5e9da7d3:::smithi17529310-1:head [copy-from ver 6] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:41.218 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:41.218 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 62.022665879999998, 2024-09-15T01:33:41.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01403071, 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 192 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192423+0000", 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3140000000000002e-06 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192431+0000", 2024-09-15T01:33:41.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2320000000000001e-06 2024-09-15T01:33:41.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192444+0000", 2024-09-15T01:33:41.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3239e-05 2024-09-15T01:33:41.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192454+0000", 2024-09-15T01:33:41.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2439999999999999e-06 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192781+0000", 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032732699999999998 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192904+0000", 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012317900000000001 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.201873+0000", 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089691839999999998 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202377+0000", 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00050339000000000004 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202392+0000", 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5201999999999999e-05 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202869+0000", 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047669800000000001 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202888+0000", 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9202e-05 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203431+0000", 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00054287499999999995 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.206451+0000", 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003020624 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 54.968111669000002, 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013669871, 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.225 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246976+0000", 2024-09-15T01:33:41.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9800000000000002e-07 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247000+0000", 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4314000000000001e-05 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247045+0000", 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4938000000000002e-05 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247053+0000", 2024-09-15T01:33:41.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247132+0000", 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8799000000000002e-05 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247142+0000", 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0511e-05 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247515+0000", 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037233299999999997 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247546+0000", 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1102000000000001e-05 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247585+0000", 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9152000000000001e-05 2024-09-15T01:33:41.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247603+0000", 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.260645+0000", 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013041662000000001 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=55, reqid=client.4477.0:517, at_version=55'45, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 44.139554162000003, 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016315308000000001, 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075533+0000", 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9199999999999999e-07 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075557+0000", 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3264e-05 2024-09-15T01:33:41.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076900+0000", 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013436749999999999 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076916+0000", 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5302999999999999e-05 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076970+0000", 2024-09-15T01:33:41.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4265999999999998e-05 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076980+0000", 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0521e-05 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077104+0000", 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012322599999999999 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077118+0000", 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4742e-05 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077164+0000", 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5972e-05 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077228+0000", 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3291000000000001e-05 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.091848+0000", 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014620055999999999 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=56, reqid=client.4477.0:518, at_version=55'46, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 44.138232527, 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015201957, 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076855+0000", 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.057e-06 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076876+0000", 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0874e-05 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079870+0000", 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0029943259999999998 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079882+0000", 2024-09-15T01:33:41.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2026e-05 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079958+0000", 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5263000000000001e-05 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079969+0000", 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.145e-05 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.080116+0000", 2024-09-15T01:33:41.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014736600000000001 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.080133+0000", 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6776e-05 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082691+0000", 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025575229999999999 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082740+0000", 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9373000000000003e-05 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.092056+0000", 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093159230000000003 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=57, reqid=client.4477.0:518, at_version=0'0, trim_to=0'0, roll_forward_to=55'46))", 2024-09-15T01:33:41.241 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 44.131308799999999, 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.77065714699999999, 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:41.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083779+0000", 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083792+0000", 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3478e-05 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853926+0000", 2024-09-15T01:33:41.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.77013421000000004 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853932+0000", 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2820000000000004e-06 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853992+0000", 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9886999999999998e-05 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854001+0000", 2024-09-15T01:33:41.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3700000000000001e-06 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854114+0000", 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000112651 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854152+0000", 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8518e-05 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854203+0000", 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1288000000000001e-05 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854223+0000", 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9884000000000001e-05 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854435+0000", 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021157399999999999 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s3 66/31 ECSubWrite(tid=130, reqid=osd.14.0:130, at_version=66'56, trim_to=0'0, roll_forward_to=60'52))", 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.134291802999996, 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016749074999999999, 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8199999999999999e-07 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080811+0000", 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6435000000000001e-05 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082265+0000", 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001453415 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082270+0000", 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9470000000000002e-06 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082610+0000", 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00034021700000000003 2024-09-15T01:33:41.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082618+0000", 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3969999999999994e-06 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084267+0000", 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001649069 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084292+0000", 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4125999999999999e-05 2024-09-15T01:33:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084507+0000", 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0002154 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084539+0000", 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2208000000000001e-05 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097544+0000", 2024-09-15T01:33:41.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013004679 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 66/54 ECSubWrite(tid=240, reqid=osd.13.0:240, at_version=66'36, trim_to=0'0, roll_forward_to=56'33))", 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.134221087, 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020188490999999999, 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4600000000000001e-07 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080879+0000", 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3557000000000001e-05 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082859+0000", 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019801799999999998 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082865+0000", 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8309999999999997e-06 2024-09-15T01:33:41.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083375+0000", 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051010200000000002 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083383+0000", 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7740000000000004e-06 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085762+0000", 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023785339999999999 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086258+0000", 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00049678099999999996 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086290+0000", 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1612999999999997e-05 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086368+0000", 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8417000000000005e-05 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101054+0000", 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014685556000000001 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.257 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=62, reqid=osd.7.0:62, at_version=66'52, trim_to=0'0, roll_forward_to=58'50))", 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081340+0000", 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.133746269, 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016057799000000001, 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081340+0000", 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081340+0000", 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081341+0000", 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3900000000000004e-07 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081364+0000", 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3078000000000002e-05 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082876+0000", 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001512298 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082880+0000", 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1150000000000004e-06 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082998+0000", 2024-09-15T01:33:41.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117386 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083007+0000", 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4849999999999998e-06 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084480+0000", 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014727799999999999 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084509+0000", 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8878e-05 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084575+0000", 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5848999999999999e-05 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085049+0000", 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047466899999999999 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097398+0000", 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012348622 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=63, reqid=osd.7.0:63, at_version=66'54, trim_to=0'0, roll_forward_to=58'50))", 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.133719030999998, 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017023607, 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.263 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081386+0000", 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952e-05 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083054+0000", 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001668295 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083059+0000", 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9359999999999998e-06 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084544+0000", 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001484983 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084582+0000", 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7764e-05 2024-09-15T01:33:41.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084807+0000", 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022473099999999999 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084975+0000", 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000167997 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086254+0000", 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001279608 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086318+0000", 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3590000000000006e-05 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.098391+0000", 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012073421000000001 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.268 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 66/31 ECSubWrite(tid=241, reqid=osd.13.0:241, at_version=66'51, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081392+0000", 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.133694202999997, 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017955004, 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081392+0000", 2024-09-15T01:33:41.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081392+0000", 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081393+0000", 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7399999999999999e-07 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081416+0000", 2024-09-15T01:33:41.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3807000000000001e-05 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083070+0000", 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001653416 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083073+0000", 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.58e-06 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083228+0000", 2024-09-15T01:33:41.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000154671 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083236+0000", 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1960000000000008e-06 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085387+0000", 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021502230000000002 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086257+0000", 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00087064300000000005 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086289+0000", 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2048000000000002e-05 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086344+0000", 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5031000000000002e-05 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.099347+0000", 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013003115000000001 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 66/31 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=66'53, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.083317+0000", 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.131769517999999, 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017650290999999999, 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083317+0000", 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083317+0000", 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083318+0000", 2024-09-15T01:33:41.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9100000000000004e-07 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083337+0000", 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9488e-05 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083371+0000", 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3522000000000002e-05 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083382+0000", 2024-09-15T01:33:41.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1715999999999999e-05 2024-09-15T01:33:41.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085431+0000", 2024-09-15T01:33:41.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020484309999999999 2024-09-15T01:33:41.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085439+0000", 2024-09-15T01:33:41.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3389999999999993e-06 2024-09-15T01:33:41.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085612+0000", 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017278700000000001 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086258+0000", 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00064593500000000004 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086290+0000", 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1813999999999997e-05 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086355+0000", 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5405000000000001e-05 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.100967+0000", 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014612263 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.279 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=64, reqid=client.4477.0:609, at_version=66'56, trim_to=0'0, roll_forward_to=66'52))", 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.087506+0000", 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.127580596000001, 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014356819999999999, 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087506+0000", 2024-09-15T01:33:41.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087506+0000", 2024-09-15T01:33:41.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087507+0000", 2024-09-15T01:33:41.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.144e-06 2024-09-15T01:33:41.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087633+0000", 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012627100000000001 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087682+0000", 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8551000000000003e-05 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087690+0000", 2024-09-15T01:33:41.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5979999999999999e-06 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087747+0000", 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7376000000000001e-05 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087758+0000", 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1065e-05 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.088106+0000", 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000347766 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.088114+0000", 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3850000000000002e-06 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091745+0000", 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003630968 2024-09-15T01:33:41.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091795+0000", 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9922000000000001e-05 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101863+0000", 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010067774 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 66/31 ECSubWrite(tid=244, reqid=osd.13.0:242, at_version=0'0, trim_to=0'0, roll_forward_to=66'53))", 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.089639+0000", 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.125447149000003, 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012039856, 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089639+0000", 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089639+0000", 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089640+0000", 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089649+0000", 2024-09-15T01:33:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0459999999999994e-06 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089681+0000", 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1588999999999999e-05 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089687+0000", 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9819999999999996e-06 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089743+0000", 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5841999999999998e-05 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089751+0000", 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3790000000000006e-06 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089878+0000", 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012716299999999999 2024-09-15T01:33:41.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091756+0000", 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018777080000000001 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091830+0000", 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4288000000000004e-05 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091848+0000", 2024-09-15T01:33:41.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7448e-05 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101679+0000", 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0098317509999999997 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 66/54 ECSubWrite(tid=243, reqid=osd.13.0:240, at_version=0'0, trim_to=0'0, roll_forward_to=66'36))", 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.125434720000001, 2024-09-15T01:33:41.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01224519, 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.48e-07 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089655+0000", 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089726+0000", 2024-09-15T01:33:41.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0436000000000001e-05 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089731+0000", 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.164e-06 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089770+0000", 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9110999999999999e-05 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089777+0000", 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2599999999999999e-06 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.090039+0000", 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000261506 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091756+0000", 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001717704 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091831+0000", 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4177000000000005e-05 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091855+0000", 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4300000000000001e-05 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101897+0000", 2024-09-15T01:33:41.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010042163 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.as0 76/69 ECSubWriteReply(tid=60, last_complete=76'39, committed=1, applied=1))", 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:18.085798+0000", 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.129288621000001, 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19697366399999999, 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085798+0000", 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085798+0000", 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085799+0000", 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9199999999999999e-07 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085804+0000", 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0440000000000003e-06 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282638+0000", 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19683383099999999 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282645+0000", 2024-09-15T01:33:41.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9800000000000001e-06 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282743+0000", 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8517999999999994e-05 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282772+0000", 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8399e-05 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.300 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/54 ECSubWrite(tid=303, reqid=client.4477.0:846, at_version=86'54, trim_to=0'0, roll_forward_to=85'52))", 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.133620+0000", 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.081466839000001, 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010527086, 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133620+0000", 2024-09-15T01:33:41.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133620+0000", 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133620+0000", 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.43e-07 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133641+0000", 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1316000000000001e-05 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133695+0000", 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3981999999999999e-05 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133702+0000", 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.849e-06 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133755+0000", 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3094999999999999e-05 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.133768+0000", 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2534000000000001e-05 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134038+0000", 2024-09-15T01:33:41.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000270571 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134053+0000", 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5091e-05 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134099+0000", 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5439999999999999e-05 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134122+0000", 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3150999999999999e-05 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.144147+0000", 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010024913999999999 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.306 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/54 ECSubWrite(tid=304, reqid=client.4477.0:847, at_version=86'55, trim_to=0'0, roll_forward_to=85'52))", 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.134900+0000", 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.080186785, 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018173871000000001, 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134900+0000", 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134900+0000", 2024-09-15T01:33:41.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134901+0000", 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0359999999999999e-06 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134932+0000", 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0868999999999999e-05 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134994+0000", 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2161000000000003e-05 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135000+0000", 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2319999999999998e-06 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135199+0000", 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000198438 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135207+0000", 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0830000000000004e-06 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135318+0000", 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110881 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135331+0000", 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3409e-05 2024-09-15T01:33:41.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135370+0000", 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9144000000000003e-05 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135466+0000", 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5527999999999997e-05 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.153074+0000", 2024-09-15T01:33:41.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01760809 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/54 ECSubWrite(tid=305, reqid=client.4477.0:848, at_version=86'56, trim_to=0'0, roll_forward_to=85'52))", 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.135237+0000", 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.07984918, 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018127175999999998, 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135237+0000", 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135237+0000", 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135238+0000", 2024-09-15T01:33:41.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3600000000000004e-07 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135253+0000", 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5339e-05 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135283+0000", 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9445e-05 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135290+0000", 2024-09-15T01:33:41.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8329999999999996e-06 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135356+0000", 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5982000000000006e-05 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135363+0000", 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7030000000000006e-06 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135446+0000", 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2695999999999996e-05 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135454+0000", 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1669999999999999e-06 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135511+0000", 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7265000000000001e-05 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135519+0000", 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6739999999999997e-06 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.153365+0000", 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017845535999999999 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.317 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/54 ECSubWrite(tid=306, reqid=client.4477.0:848, at_version=0'0, trim_to=0'0, roll_forward_to=86'56))", 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.136794+0000", 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.078292415, 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016821658999999999, 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136794+0000", 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136794+0000", 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136795+0000", 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0890000000000001e-06 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136806+0000", 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0301e-05 2024-09-15T01:33:41.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136844+0000", 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8002e-05 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136852+0000", 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3979999999999999e-06 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136958+0000", 2024-09-15T01:33:41.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010578600000000001 2024-09-15T01:33:41.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136969+0000", 2024-09-15T01:33:41.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0767999999999999e-05 2024-09-15T01:33:41.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137111+0000", 2024-09-15T01:33:41.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142922 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137120+0000", 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137317+0000", 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019695499999999999 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137331+0000", 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.385e-05 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.153616+0000", 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016284666 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.323 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 86/31 ECSubWrite(tid=104, reqid=client.4477.0:836, at_version=0'0, trim_to=0'0, roll_forward_to=86'87))", 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.137961+0000", 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.077125390999999, 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015483966, 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137961+0000", 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137961+0000", 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137962+0000", 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4200000000000004e-07 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137975+0000", 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2993000000000001e-05 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138014+0000", 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9050000000000001e-05 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138021+0000", 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1130000000000003e-06 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138076+0000", 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4867999999999997e-05 2024-09-15T01:33:41.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138087+0000", 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0623e-05 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138238+0000", 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000150944 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.139771+0000", 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00153374 2024-09-15T01:33:41.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.139849+0000", 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7737999999999993e-05 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.139880+0000", 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0809999999999998e-05 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.153445+0000", 2024-09-15T01:33:41.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013565145000000001 2024-09-15T01:33:41.329 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.329 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.329 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.329 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.329 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.329 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:41.330 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:33:41.331 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:41.329+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:41.431 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:41.429+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:41.471 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:41.471 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:41.480 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:33:41.532 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:41.531+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:41.576 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:41.576 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:41.576 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:41.576 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:41.576 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:41.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:33:41.633 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:41.632+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:41.683 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:41.683 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:41.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:41.683 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:41.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.684 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:108 3.ds0 3:b063f44c:::smithi17529310-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:41.684 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:41.684 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.434106611999994, 2024-09-15T01:33:41.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013756292, 2024-09-15T01:33:41.684 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.684 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:41.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.684 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 108 2024-09-15T01:33:41.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.684 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:41.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:41.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246459+0000", 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6800000000000001e-07 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246461+0000", 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.131e-06 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246468+0000", 2024-09-15T01:33:41.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5529999999999998e-06 2024-09-15T01:33:41.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246475+0000", 2024-09-15T01:33:41.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5830000000000001e-06 2024-09-15T01:33:41.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.248249+0000", 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017739990000000001 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.248333+0000", 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4288000000000003e-05 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.249154+0000", 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00082014900000000005 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.249218+0000", 2024-09-15T01:33:41.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4519999999999999e-05 2024-09-15T01:33:41.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.260215+0000", 2024-09-15T01:33:41.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010996601 2024-09-15T01:33:41.689 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.689 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.689 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.690 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:41.690 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:41.690 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.439342109000002, 2024-09-15T01:33:41.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014051065, 2024-09-15T01:33:41.690 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.690 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.690 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:41.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241243+0000", 2024-09-15T01:33:41.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0126e-05 2024-09-15T01:33:41.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241422+0000", 2024-09-15T01:33:41.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017906899999999999 2024-09-15T01:33:41.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241429+0000", 2024-09-15T01:33:41.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4139999999999999e-06 2024-09-15T01:33:41.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241500+0000", 2024-09-15T01:33:41.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1098000000000003e-05 2024-09-15T01:33:41.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241510+0000", 2024-09-15T01:33:41.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8470000000000008e-06 2024-09-15T01:33:41.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241787+0000", 2024-09-15T01:33:41.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027729700000000003 2024-09-15T01:33:41.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242007+0000", 2024-09-15T01:33:41.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000220134 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242146+0000", 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013846799999999999 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242278+0000", 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0001326 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.255274+0000", 2024-09-15T01:33:41.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01299575 2024-09-15T01:33:41.696 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.696 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.696 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.696 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:41.696 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:41.696 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.436883764000001, 2024-09-15T01:33:41.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014188612, 2024-09-15T01:33:41.697 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.697 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.697 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:41.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:41.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243682+0000", 2024-09-15T01:33:41.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0299999999999999e-07 2024-09-15T01:33:41.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243699+0000", 2024-09-15T01:33:41.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7575999999999999e-05 2024-09-15T01:33:41.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244099+0000", 2024-09-15T01:33:41.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039909500000000003 2024-09-15T01:33:41.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244108+0000", 2024-09-15T01:33:41.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3209999999999994e-06 2024-09-15T01:33:41.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244168+0000", 2024-09-15T01:33:41.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9688999999999997e-05 2024-09-15T01:33:41.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244180+0000", 2024-09-15T01:33:41.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2031000000000001e-05 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245504+0000", 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013241520000000001 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245537+0000", 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3287000000000002e-05 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245645+0000", 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108435 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245664+0000", 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8329999999999999e-05 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.257870+0000", 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012206092999999999 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.702 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.433600181000003, 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013797696999999999, 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246966+0000", 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000002e-07 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246979+0000", 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3111e-05 2024-09-15T01:33:41.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247017+0000", 2024-09-15T01:33:41.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8096999999999998e-05 2024-09-15T01:33:41.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247025+0000", 2024-09-15T01:33:41.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9100000000000005e-06 2024-09-15T01:33:41.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247095+0000", 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0049000000000006e-05 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247104+0000", 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7919999999999998e-06 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247217+0000", 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011299 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247376+0000", 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015968800000000001 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247417+0000", 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0228999999999997e-05 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248151+0000", 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00073475999999999999 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.260763+0000", 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012611282999999999 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:283 3.ds0 3:bfa066a3:::smithi17529310-39:head [copy-from ver 2] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.421799442999998, 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015911307999999999, 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.708 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 283 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258768+0000", 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0980000000000001e-06 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258779+0000", 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1272e-05 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258795+0000", 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5679999999999999e-05 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258802+0000", 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9029999999999998e-06 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258879+0000", 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7035999999999999e-05 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.259139+0000", 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260456 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268247+0000", 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0091076109999999998 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268518+0000", 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027146499999999998 2024-09-15T01:33:41.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268542+0000", 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3601e-05 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268616+0000", 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3944999999999998e-05 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.269087+0000", 2024-09-15T01:33:41.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047095399999999999 2024-09-15T01:33:41.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:41.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.274577+0000", 2024-09-15T01:33:41.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0054903019999999999 2024-09-15T01:33:41.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.274677+0000", 2024-09-15T01:33:41.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9984999999999999e-05 2024-09-15T01:33:41.715 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.715 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.715 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.716 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 44/31 ECSubWriteReply(tid=43, last_complete=44'32, committed=1, applied=1))", 2024-09-15T01:33:41.716 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:41.716 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.405118577000003, 2024-09-15T01:33:41.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.192594981, 2024-09-15T01:33:41.716 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.716 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.717 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:41.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:41.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275447+0000", 2024-09-15T01:33:41.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2099999999999995e-07 2024-09-15T01:33:41.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275452+0000", 2024-09-15T01:33:41.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2050000000000004e-06 2024-09-15T01:33:41.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.467889+0000", 2024-09-15T01:33:41.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19243722999999999 2024-09-15T01:33:41.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.467922+0000", 2024-09-15T01:33:41.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3237e-05 2024-09-15T01:33:41.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.468017+0000", 2024-09-15T01:33:41.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4543999999999995e-05 2024-09-15T01:33:41.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.468041+0000", 2024-09-15T01:33:41.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4844000000000001e-05 2024-09-15T01:33:41.724 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.724 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.725 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.725 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 66/31 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=66'53, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:41.726 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:41.726 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.597883351999997, 2024-09-15T01:33:41.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014557652000000001, 2024-09-15T01:33:41.726 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.726 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.727 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:41.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:41.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082683+0000", 2024-09-15T01:33:41.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2999999999999999e-07 2024-09-15T01:33:41.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082704+0000", 2024-09-15T01:33:41.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1381000000000001e-05 2024-09-15T01:33:41.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082736+0000", 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2239000000000001e-05 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082743+0000", 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6449999999999999e-06 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082927+0000", 2024-09-15T01:33:41.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018380800000000001 2024-09-15T01:33:41.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082934+0000", 2024-09-15T01:33:41.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1600000000000001e-06 2024-09-15T01:33:41.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083133+0000", 2024-09-15T01:33:41.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019963000000000001 2024-09-15T01:33:41.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083536+0000", 2024-09-15T01:33:41.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040219000000000002 2024-09-15T01:33:41.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083587+0000", 2024-09-15T01:33:41.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1600000000000001e-05 2024-09-15T01:33:41.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083644+0000", 2024-09-15T01:33:41.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6753999999999999e-05 2024-09-15T01:33:41.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097239+0000", 2024-09-15T01:33:41.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013595415 2024-09-15T01:33:41.733 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.734 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.734 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.734 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s5 71/69 ECSubWrite(tid=73, reqid=client.4477.0:701, at_version=71'45, trim_to=0'0, roll_forward_to=70'43))", 2024-09-15T01:33:41.734 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.321241+0000", 2024-09-15T01:33:41.734 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.359323616000001, 2024-09-15T01:33:41.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014395221999999999, 2024-09-15T01:33:41.735 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.735 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.735 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321241+0000", 2024-09-15T01:33:41.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321241+0000", 2024-09-15T01:33:41.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321242+0000", 2024-09-15T01:33:41.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.36e-07 2024-09-15T01:33:41.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327486+0000", 2024-09-15T01:33:41.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0062439130000000002 2024-09-15T01:33:41.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327562+0000", 2024-09-15T01:33:41.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6878999999999994e-05 2024-09-15T01:33:41.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327570+0000", 2024-09-15T01:33:41.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3520000000000001e-06 2024-09-15T01:33:41.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327617+0000", 2024-09-15T01:33:41.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6839999999999999e-05 2024-09-15T01:33:41.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327627+0000", 2024-09-15T01:33:41.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.927e-06 2024-09-15T01:33:41.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328144+0000", 2024-09-15T01:33:41.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051762900000000003 2024-09-15T01:33:41.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328237+0000", 2024-09-15T01:33:41.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3153999999999996e-05 2024-09-15T01:33:41.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328549+0000", 2024-09-15T01:33:41.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000311873 2024-09-15T01:33:41.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328572+0000", 2024-09-15T01:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2751999999999998e-05 2024-09-15T01:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.335637+0000", 2024-09-15T01:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0070647669999999996 2024-09-15T01:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.744 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.744 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.744 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:823 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [copy-from ver 45] snapc 30=[30,2e,2c,2b,29,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-15T01:33:41.744 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:27.126788+0000", 2024-09-15T01:33:41.744 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.553777314, 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.02620683, 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 823 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126788+0000", 2024-09-15T01:33:41.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126788+0000", 2024-09-15T01:33:41.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126790+0000", 2024-09-15T01:33:41.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9360000000000002e-06 2024-09-15T01:33:41.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126797+0000", 2024-09-15T01:33:41.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0330000000000003e-06 2024-09-15T01:33:41.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126824+0000", 2024-09-15T01:33:41.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7637e-05 2024-09-15T01:33:41.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126834+0000", 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.268e-06 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126932+0000", 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7882999999999998e-05 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127109+0000", 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017735299999999999 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.142551+0000", 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015441939 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143350+0000", 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00079953299999999995 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143360+0000", 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0098999999999999e-05 2024-09-15T01:33:41.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143663+0000", 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030244499999999998 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143835+0000", 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017216200000000001 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.152928+0000", 2024-09-15T01:33:41.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0090929219999999998 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.152995+0000", 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6619999999999996e-05 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:824 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.553719259999999, 2024-09-15T01:33:41.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016980479, 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 824 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6199999999999998e-07 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126849+0000", 2024-09-15T01:33:41.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6800000000000002e-06 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127309+0000", 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045968300000000001 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127316+0000", 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9500000000000004e-06 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127358+0000", 2024-09-15T01:33:41.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1990999999999998e-05 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127413+0000", 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5448999999999997e-05 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143574+0000", 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016161045999999998 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143678+0000", 2024-09-15T01:33:41.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103719 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143826+0000", 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014839900000000001 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:834 3.ds0 3:bfa066a3:::smithi17529310-39:head [append 0~475136] snapc 32=[32,30,2e,2c,2b,29,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.122940+0000", 2024-09-15T01:33:41.756 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.557625127, 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01465559, 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 834 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.122940+0000", 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.122940+0000", 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.122941+0000", 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.187e-06 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123093+0000", 2024-09-15T01:33:41.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000152087 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123103+0000", 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4339999999999998e-06 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123108+0000", 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123144+0000", 2024-09-15T01:33:41.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5818000000000002e-05 2024-09-15T01:33:41.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123231+0000", 2024-09-15T01:33:41.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7831000000000007e-05 2024-09-15T01:33:41.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123712+0000", 2024-09-15T01:33:41.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00048043899999999999 2024-09-15T01:33:41.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123895+0000", 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018332600000000001 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123903+0000", 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8849999999999999e-06 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.124026+0000", 2024-09-15T01:33:41.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012307799999999999 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.124343+0000", 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031695200000000001 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137541+0000", 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013197690999999999 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137596+0000", 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4832e-05 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:837 3.ds0 3:bfa066a3:::smithi17529310-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.125675+0000", 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.554890410000001, 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016649648999999999, 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.763 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 837 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125675+0000", 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125675+0000", 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125675+0000", 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6499999999999999e-07 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125678+0000", 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0800000000000002e-06 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125688+0000", 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9580000000000002e-06 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125694+0000", 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4670000000000003e-06 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.127414+0000", 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017203640000000001 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:41.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.127494+0000", 2024-09-15T01:33:41.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0179999999999995e-05 2024-09-15T01:33:41.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138114+0000", 2024-09-15T01:33:41.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010619161 2024-09-15T01:33:41.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138176+0000", 2024-09-15T01:33:41.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2644999999999994e-05 2024-09-15T01:33:41.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.142324+0000", 2024-09-15T01:33:41.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0041481290000000004 2024-09-15T01:33:41.768 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.769 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.769 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.769 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:842 3.ds0 3:b063f44c:::smithi17529310-36:head [read 0~2949120,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:41.769 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.125725+0000", 2024-09-15T01:33:41.769 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.554840464, 2024-09-15T01:33:41.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013984874, 2024-09-15T01:33:41.770 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.770 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:41.770 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.770 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.770 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.770 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 842 2024-09-15T01:33:41.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.770 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125725+0000", 2024-09-15T01:33:41.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125725+0000", 2024-09-15T01:33:41.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125725+0000", 2024-09-15T01:33:41.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1800000000000001e-07 2024-09-15T01:33:41.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125728+0000", 2024-09-15T01:33:41.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:33:41.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125734+0000", 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6789999999999999e-06 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125738+0000", 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.126e-06 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.128322+0000", 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025834970000000001 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.128409+0000", 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7362999999999997e-05 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.139709+0000", 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01130014 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:843 3.ds0 3:b063f44c:::smithi17529310-36:head [read 0~2949120,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.125742+0000", 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.554822958000001, 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015933147000000002, 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 843 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125742+0000", 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125742+0000", 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125742+0000", 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9900000000000002e-07 2024-09-15T01:33:41.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125745+0000", 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2900000000000001e-06 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125751+0000", 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0310000000000004e-06 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125755+0000", 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8190000000000002e-06 2024-09-15T01:33:41.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.129247+0000", 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0034922130000000001 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.129335+0000", 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8098999999999998e-05 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.141675+0000", 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012340396 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 90/31 ECSubWriteReply(tid=117, last_complete=90'103, committed=1, applied=1))", 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:32.087896+0000", 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.5926694189999999, 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.78023241700000001, 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:41.781 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.087896+0000", 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.087896+0000", 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.087896+0000", 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8899999999999999e-07 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.087900+0000", 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0180000000000003e-06 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.867952+0000", 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.78005183600000005 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.867963+0000", 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1214e-05 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.868082+0000", 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118872 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.868128+0000", 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5887999999999998e-05 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:888 3.ds0 3:b50d79c0:::smithi17529310-18:head [read 0~2228224,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:33.082507+0000", 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.5980582949999995, 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024728772999999999, 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 888 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.785 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082507+0000", 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082507+0000", 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082507+0000", 2024-09-15T01:33:41.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.46e-07 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082509+0000", 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1339999999999998e-06 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082514+0000", 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7779999999999998e-06 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082518+0000", 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2289999999999997e-06 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082626+0000", 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107278 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082630+0000", 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3000000000000003e-06 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.788 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.085853+0000", 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0032233330000000001 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.085981+0000", 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127572 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.107236+0000", 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021254802999999999 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:898 3.ds0 3:b063f44c:::smithi17529310-36:19 [read 0~589824,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:33.083177+0000", 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.5973880299999994, 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024976851000000001, 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 898 2024-09-15T01:33:41.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083177+0000", 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083177+0000", 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083178+0000", 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.006e-06 2024-09-15T01:33:41.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083184+0000", 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9150000000000001e-06 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083199+0000", 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4789e-05 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083205+0000", 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2330000000000003e-06 2024-09-15T01:33:41.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083312+0000", 2024-09-15T01:33:41.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010702899999999999 2024-09-15T01:33:41.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:41.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083319+0000", 2024-09-15T01:33:41.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0319999999999998e-06 2024-09-15T01:33:41.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.086360+0000", 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030408840000000002 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.086443+0000", 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2886999999999994e-05 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.108154+0000", 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021711075999999999 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.794 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:899 3.ds0 3:b063f44c:::smithi17529310-36:19 [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:33.083217+0000", 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.5973477569999996, 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.025600806, 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 899 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083217+0000", 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083217+0000", 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083218+0000", 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5199999999999997e-07 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083221+0000", 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9249999999999999e-06 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083228+0000", 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5340000000000002e-06 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083234+0000", 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1970000000000004e-06 2024-09-15T01:33:41.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083361+0000", 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127136 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083366+0000", 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8949999999999997e-06 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.093115+0000", 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0097499279999999997 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.093210+0000", 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4549000000000006e-05 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.108818+0000", 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01560809 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:900 3.ds0 3:b063f44c:::smithi17529310-36:19 [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:33.083238+0000", 2024-09-15T01:33:41.799 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.5973267670000002, 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.026108421, 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 900 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083238+0000", 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083238+0000", 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083239+0000", 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5200000000000002e-07 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083242+0000", 2024-09-15T01:33:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0280000000000001e-06 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083248+0000", 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6900000000000003e-06 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083253+0000", 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1280000000000001e-06 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083395+0000", 2024-09-15T01:33:41.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014244099999999999 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083403+0000", 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8169999999999999e-06 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.093539+0000", 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010135933999999999 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.093696+0000", 2024-09-15T01:33:41.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000157427 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.109347+0000", 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015650503999999999 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:916 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [copy-from ver 60] snapc 39=[39,34,32,2e,2c,29,1f,19] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:35.104005+0000", 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.576560454, 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013890008000000001, 2024-09-15T01:33:41.804 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 916 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104005+0000", 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104005+0000", 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104007+0000", 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1859999999999999e-06 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104018+0000", 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1613e-05 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104040+0000", 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1548000000000001e-05 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104049+0000", 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9209999999999998e-06 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104099+0000", 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0553000000000002e-05 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104180+0000", 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0117999999999994e-05 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.111941+0000", 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0077609970000000004 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.112308+0000", 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036721600000000002 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:41.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.112317+0000", 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8650000000000007e-06 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.112391+0000", 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4833999999999995e-05 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.112420+0000", 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8977999999999999e-05 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.114820+0000", 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023993360000000002 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.117895+0000", 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030748429999999998 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:41.810 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:41.812 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:33:41.812 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:41.810+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:41.908 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:41.908 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:41.908 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:41.908 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:41.912 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:41.910+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:41.917 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:33:41.948 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:33:42.014 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:42.012+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:42.053 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:42.054 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:42.054 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:42.054 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:42.054 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:42.062 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:33:42.086 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:42.086 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:42.089 INFO:tasks.rados.rados.0.smithi175.stdout:382: done (1 left) 2024-09-15T01:33:42.090 INFO:tasks.rados.rados.0.smithi175.stdout:383: read oid 6 snap -1 2024-09-15T01:33:42.090 INFO:tasks.rados.rados.0.smithi175.stdout:383: expect (ObjNum 105 snap 10 seq_num 105) 2024-09-15T01:33:42.090 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 16 current snap is 34 2024-09-15T01:33:42.090 INFO:tasks.rados.rados.0.smithi175.stdout:384: seq_num 155 ranges {6406144=212992} 2024-09-15T01:33:42.092 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2024-09-15T01:33:42.092 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2024-09-15T01:33:42.114 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:42.113+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:42.117 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:42.117 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:42.126 INFO:tasks.rados.rados.0.smithi175.stdout:384: writing smithi17529310-16 from 6406144 to 6619136 tid 1 2024-09-15T01:33:42.126 INFO:tasks.rados.rados.0.smithi175.stdout:385: setattr oid 29 current snap is 34 2024-09-15T01:33:42.126 INFO:tasks.rados.rados.0.smithi175.stdout:381: expect (ObjNum 124 snap 20 seq_num 124) 2024-09-15T01:33:42.132 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:33:42.138 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:42.138 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:42.157 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:42.158 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:42.182 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:42.182 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:42.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:42.182 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:42.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:99 3.fs0 3:f630fe6e:::smithi17529310-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.944671075000002, 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055995037999999997, 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235203+0000", 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4000000000000002e-07 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235205+0000", 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3e-06 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235212+0000", 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4389999999999997e-06 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235216+0000", 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.031e-06 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.236877+0000", 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00166088 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.236933+0000", 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6873999999999998e-05 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255906+0000", 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018972066999999999 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255957+0000", 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1467e-05 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291198+0000", 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035240540000000001 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:114 3.fs0 3:f8072b57:::smithi17529310-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.926231154000007, 2024-09-15T01:33:42.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048699039999999999, 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 114 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253643+0000", 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6899999999999998e-07 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253646+0000", 2024-09-15T01:33:42.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253652+0000", 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3369999999999996e-06 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253656+0000", 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.031e-06 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255410+0000", 2024-09-15T01:33:42.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017543420000000001 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255505+0000", 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4704999999999995e-05 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290833+0000", 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035327711999999997 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290893+0000", 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0384999999999999e-05 2024-09-15T01:33:42.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.302341+0000", 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011448274 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:145 3.cs0 3:34a45b5f:::smithi17529310-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.882004585999994, 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15351202899999999, 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.192 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 145 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:42.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0500000000000004e-07 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297871+0000", 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9769999999999999e-06 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297878+0000", 2024-09-15T01:33:42.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9120000000000001e-06 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297882+0000", 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0910000000000003e-06 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.298932+0000", 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00104904 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:42.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.299009+0000", 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7095000000000006e-05 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.306936+0000", 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0079278730000000002 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.307022+0000", 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5254999999999996e-05 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.451381+0000", 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14435928100000001 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:312 3.cs0 3:37dffec7:::smithi17529310-13:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.901680970999998, 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770305000000002, 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.197 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 312 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0200000000000002e-07 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278196+0000", 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7319999999999998e-06 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278216+0000", 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0375999999999999e-05 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278221+0000", 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4889999999999997e-06 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279668+0000", 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014473929999999999 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279753+0000", 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4567999999999994e-05 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280213+0000", 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00046030399999999998 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280327+0000", 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011404399999999999 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280342+0000", 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4403000000000001e-05 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280467+0000", 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000125594 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280562+0000", 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5317999999999998e-05 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328870+0000", 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048307782 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328963+0000", 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2700000000000004e-05 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:313 3.cs0 3:37dffec7:::smithi17529310-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.901648090000002, 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053267040000000002, 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.203 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 313 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278226+0000", 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7100000000000002e-07 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278228+0000", 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1859999999999999e-06 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278235+0000", 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9020000000000002e-06 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278240+0000", 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9540000000000003e-06 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280404+0000", 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021641469999999999 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280503+0000", 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9090000000000002e-05 2024-09-15T01:33:42.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328996+0000", 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048492533999999997 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.329053+0000", 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6997999999999999e-05 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.331493+0000", 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024397580000000002 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.207 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=95, last_complete=44'34, committed=1, applied=1))", 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.898243747000002, 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047344538999999998, 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:42.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7700000000000001e-07 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281632+0000", 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.562e-06 2024-09-15T01:33:42.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328775+0000", 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047143813 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328779+0000", 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.895e-06 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328803+0000", 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4076000000000001e-05 2024-09-15T01:33:42.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328974+0000", 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017101599999999999 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:324 3.cs0 3:34a45b5f:::smithi17529310-48:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.84922864, 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18862163400000001, 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.211 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 324 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:42.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699999999999999e-07 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330648+0000", 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.694e-06 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330654+0000", 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4090000000000002e-06 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330659+0000", 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3210000000000004e-06 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332198+0000", 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015388699999999999 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332262+0000", 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4196000000000007e-05 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332785+0000", 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052292099999999998 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332870+0000", 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5049999999999994e-05 2024-09-15T01:33:42.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332877+0000", 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7769999999999997e-06 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332902+0000", 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5012e-05 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332962+0000", 2024-09-15T01:33:42.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0649999999999997e-05 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519220+0000", 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18625739199999999 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519267+0000", 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6875000000000001e-05 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:325 3.cs0 3:34a45b5f:::smithi17529310-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.849210577000001, 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.189890015, 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 325 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330667+0000", 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4410000000000002e-06 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330673+0000", 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5429999999999999e-06 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330676+0000", 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.683e-06 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333439+0000", 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002763415 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333492+0000", 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2760000000000003e-05 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519496+0000", 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18600408199999999 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519560+0000", 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3660999999999999e-05 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.520553+0000", 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00099301399999999996 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=100, last_complete=44'39, committed=1, applied=1))", 2024-09-15T01:33:42.222 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.846808533999997, 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18640773799999999, 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:42.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5999999999999999e-08 2024-09-15T01:33:42.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333066+0000", 2024-09-15T01:33:42.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.043e-06 2024-09-15T01:33:42.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519088+0000", 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18602205399999999 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519092+0000", 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0890000000000002e-06 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519171+0000", 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8857999999999996e-05 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519473+0000", 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030159799999999998 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 47/31 ECSubWriteReply(tid=113, last_complete=47'51, committed=1, applied=1))", 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.919986723000001, 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.63648575600000001, 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.226 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-07 2024-09-15T01:33:42.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259890+0000", 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5859999999999999e-06 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.895924+0000", 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.63603432699999995 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.895929+0000", 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1340000000000003e-06 2024-09-15T01:33:42.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.896000+0000", 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0687999999999998e-05 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.896373+0000", 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037265200000000001 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.229 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 48/31 ECSubWriteReply(tid=128, last_complete=48'57, committed=1, applied=1))", 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.873171233999997, 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18111660500000001, 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306703+0000", 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3e-07 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306707+0000", 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7500000000000001e-06 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487741+0000", 2024-09-15T01:33:42.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18103418199999999 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487745+0000", 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2189999999999998e-06 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487789+0000", 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3430000000000003e-05 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487819+0000", 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0394000000000001e-05 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 53/31 ECSubWriteReply(tid=146, last_complete=53'74, committed=1, applied=1))", 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.979329784999997, 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.74248160299999999, 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-07 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200550+0000", 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5910000000000003e-06 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.942942+0000", 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.74239190499999996 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.942965+0000", 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2983999999999999e-05 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.943000+0000", 2024-09-15T01:33:42.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5336999999999997e-05 2024-09-15T01:33:42.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.943025+0000", 2024-09-15T01:33:42.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5250999999999999e-05 2024-09-15T01:33:42.236 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.236 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.236 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.7s6 55/52 [PushOp(3:e6e8875f:::smithi17529310-31:head, version: 50'31, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:e6e8875f:::smithi17529310-31:head@50'31, size: 1949696, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 45.08161518, 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18071695199999999, 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:42.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098259+0000", 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0200000000000001e-07 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098545+0000", 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028574699999999999 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278693+0000", 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.180148009 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278700+0000", 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.888e-06 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278803+0000", 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102766 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278975+0000", 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017284000000000001 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:524 3.fs0 3:f8e751c8:::smithi17529310-17:head [setxattr _header (57) in=7b] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:42.240 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 44.085850706999999, 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.115556992, 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 524 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2200000000000004e-07 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094028+0000", 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9710000000000003e-06 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094040+0000", 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1066e-05 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094044+0000", 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8770000000000001e-06 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095122+0000", 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010771839999999999 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095188+0000", 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6298999999999997e-05 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095521+0000", 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033339600000000002 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095614+0000", 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2295999999999999e-05 2024-09-15T01:33:42.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095620+0000", 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8000000000000001e-06 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095653+0000", 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2867000000000002e-05 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095740+0000", 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6300999999999999e-05 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209497+0000", 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.113757184 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209580+0000", 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3128999999999996e-05 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:525 3.fs0 3:f8e751c8:::smithi17529310-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 44.085824881999997, 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11664384, 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 525 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:42.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6100000000000001e-07 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094051+0000", 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1950000000000002e-06 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094058+0000", 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4350000000000002e-06 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094063+0000", 2024-09-15T01:33:42.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2039999999999996e-06 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095664+0000", 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001600642 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095725+0000", 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1707999999999999e-05 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209923+0000", 2024-09-15T01:33:42.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.114197744 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209997+0000", 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3341000000000001e-05 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.210693+0000", 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00069601000000000005 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:691 3.fs0 3:f73c1a72:::smithi17529310-29:head [copy-from ver 48] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.296702+0000", 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.883171304000001, 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.042728760999999997, 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 691 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296702+0000", 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296702+0000", 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296703+0000", 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296704+0000", 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7039999999999999e-06 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296708+0000", 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0439999999999997e-06 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296710+0000", 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0200000000000001e-06 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.299482+0000", 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027719820000000001 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.299578+0000", 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5361999999999999e-05 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321672+0000", 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022094721000000001 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322236+0000", 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00056322499999999999 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322250+0000", 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.438e-05 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322630+0000", 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00038026599999999998 2024-09-15T01:33:42.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322764+0000", 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013368899999999999 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.339358+0000", 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016593988 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.339431+0000", 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3041999999999996e-05 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:746 3.9s0 3:98b2f2fd:::smithi17529310-14:head [append 0~0] snapc 27=[27,26,24,22,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:18.078177+0000", 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 24.101696427, 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0091383140000001, 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.258 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 746 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078177+0000", 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078177+0000", 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078178+0000", 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1799999999999995e-07 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078181+0000", 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.038e-06 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078187+0000", 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4760000000000003e-06 2024-09-15T01:33:42.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078191+0000", 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7189999999999999e-06 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078448+0000", 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025739300000000002 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op must wait for map", 2024-09-15T01:33:42.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078457+0000", 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7590000000000003e-06 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.082114+0000", 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036565579999999999 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.082117+0000", 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0299999999999998e-06 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.084154+0000", 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0020370750000001 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.084312+0000", 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015769300000000001 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.084835+0000", 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052329899999999996 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.085132+0000", 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029761499999999999 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.085143+0000", 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0951000000000001e-05 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.085193+0000", 2024-09-15T01:33:42.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9369000000000001e-05 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.086029+0000", 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00083588200000000003 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.086781+0000", 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00075188200000000005 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087315+0000", 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00053495699999999997 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:747 3.9s0 3:98b2f2fd:::smithi17529310-14:head [rollback 19] snapc 27=[27,26,24,22,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 24.101400033000001, 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.011011909, 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:42.266 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 747 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:42.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8299999999999995e-07 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078479+0000", 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5549999999999996e-06 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078489+0000", 2024-09-15T01:33:42.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0339999999999999e-05 2024-09-15T01:33:42.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078495+0000", 2024-09-15T01:33:42.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4999999999999999e-06 2024-09-15T01:33:42.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078537+0000", 2024-09-15T01:33:42.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2194999999999998e-05 2024-09-15T01:33:42.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078544+0000", 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0759999999999997e-06 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.084273+0000", 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057284170000000004 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.084275+0000", 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4049999999999998e-06 2024-09-15T01:33:42.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087195+0000", 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.002919946 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087315+0000", 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000119883 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087944+0000", 2024-09-15T01:33:42.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00062966699999999999 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088093+0000", 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000148531 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088104+0000", 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1381e-05 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088151+0000", 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6495000000000001e-05 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088666+0000", 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051494400000000004 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.089420+0000", 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00075379099999999997 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.089485+0000", 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5900000000000003e-05 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 81/76 ECSubWriteReply(tid=313, last_complete=81'63, committed=1, applied=1))", 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:23.083561+0000", 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.09631216, 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87779885599999996, 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083561+0000", 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083561+0000", 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083562+0000", 2024-09-15T01:33:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5400000000000002e-07 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083564+0000", 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4739999999999999e-06 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.960927+0000", 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87736222600000002 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.960933+0000", 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3550000000000001e-06 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.961015+0000", 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1646999999999999e-05 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.961360+0000", 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0003457 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 87/31 ECSubWriteReply(tid=365, last_complete=87'192, committed=1, applied=1))", 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:29.144736+0000", 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.035137977, 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.52162740600000002, 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.144736+0000", 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.144736+0000", 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.144736+0000", 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2900000000000004e-07 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.144740+0000", 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4300000000000002e-06 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.665922+0000", 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.52118211400000003 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.665927+0000", 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3469999999999998e-06 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.666003+0000", 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6185000000000003e-05 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.666363+0000", 2024-09-15T01:33:42.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035980099999999998 2024-09-15T01:33:42.281 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.281 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.281 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.281 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.281 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.281 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:42.282 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:42.282 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:42.283 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:42.283 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:42.284 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-15T01:33:42.284 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:42.284 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:42.285 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:42.285 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:42.285 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:42.285 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:42.286 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:42.281+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:42.296 INFO:tasks.rados.rados.0.smithi175.stdout:383: expect (ObjNum 105 snap 10 seq_num 105) 2024-09-15T01:33:42.312 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:42.312 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:42.344 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:42.345 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:42.379 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:42.383 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:42.381+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:42.393 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:42.393 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:42.393 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:42.393 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:42.402 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-15T01:33:42.412 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:42.412 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:42.424 INFO:tasks.rados.rados.0.smithi175.stdout:384: finishing write tid 1 to smithi17529310-16 2024-09-15T01:33:42.425 INFO:tasks.rados.rados.0.smithi175.stdout:384: finishing write tid 2 to smithi17529310-16 2024-09-15T01:33:42.425 INFO:tasks.rados.rados.0.smithi175.stdout:384: finishing write tid 3 to smithi17529310-16 2024-09-15T01:33:42.425 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 16 v 136 (ObjNum 155 snap 34 seq_num 155) dirty exists 2024-09-15T01:33:42.425 INFO:tasks.rados.rados.0.smithi175.stdout:384: left oid 16 (ObjNum 155 snap 34 seq_num 155) 2024-09-15T01:33:42.425 INFO:tasks.rados.rados.0.smithi175.stdout:381: done (3 left) 2024-09-15T01:33:42.425 INFO:tasks.rados.rados.0.smithi175.stdout:383: done (2 left) 2024-09-15T01:33:42.425 INFO:tasks.rados.rados.0.smithi175.stdout:384: done (1 left) 2024-09-15T01:33:42.425 INFO:tasks.rados.rados.0.smithi175.stdout:386: read oid 39 snap -1 2024-09-15T01:33:42.425 INFO:tasks.rados.rados.0.smithi175.stdout:386: expect (ObjNum 139 snap 29 seq_num 139) 2024-09-15T01:33:42.425 INFO:tasks.rados.rados.0.smithi175.stdout:387: snap_remove snap 12 2024-09-15T01:33:42.447 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:42.447 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:42.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:42.446+0000 7f7d5908a640 0 -- 172.21.15.42:0/370593127 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7f7d54192910 msgr2=0x7f7d54194cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-15T01:33:42.484 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-15T01:33:42.481+0000 7f1a3f7f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28828) UID: 0 2024-09-15T01:33:42.501 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:42.502 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:42.502 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:42.502 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:42.502 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:42.510 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-15T01:33:42.595 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:42.593+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:42.650 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:42.650 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:42.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:42.650 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:42.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.650 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:11 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:42.650 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:42.650 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.470133946000004, 2024-09-15T01:33:42.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032033109999999997, 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 11 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176990+0000", 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2200000000000002e-07 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176995+0000", 2024-09-15T01:33:42.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8040000000000001e-06 2024-09-15T01:33:42.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177006+0000", 2024-09-15T01:33:42.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1276e-05 2024-09-15T01:33:42.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177012+0000", 2024-09-15T01:33:42.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9370000000000001e-06 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177055+0000", 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2472999999999998e-05 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177197+0000", 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142519 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177567+0000", 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036945000000000001 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177659+0000", 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1928999999999993e-05 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177694+0000", 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4885000000000002e-05 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177722+0000", 2024-09-15T01:33:42.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8334e-05 2024-09-15T01:33:42.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177733+0000", 2024-09-15T01:33:42.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1491000000000001e-05 2024-09-15T01:33:42.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:42.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208963+0000", 2024-09-15T01:33:42.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031230106000000001 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209022+0000", 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8984000000000003e-05 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:12 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:42.657 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.469383678, 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033440417, 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 12 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2500000000000005e-07 2024-09-15T01:33:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177745+0000", 2024-09-15T01:33:42.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7269999999999998e-06 2024-09-15T01:33:42.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177755+0000", 2024-09-15T01:33:42.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9000000000000001e-06 2024-09-15T01:33:42.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177760+0000", 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1150000000000002e-06 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177801+0000", 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0802e-05 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177877+0000", 2024-09-15T01:33:42.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6604e-05 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209067+0000", 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031189505999999999 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209144+0000", 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6728000000000001e-05 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211180+0000", 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020363099999999999 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=12, reqid=client.4477.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.457179464999996, 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022201101000000001, 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.663 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:42.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.75e-07 2024-09-15T01:33:42.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189957+0000", 2024-09-15T01:33:42.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2531000000000001e-05 2024-09-15T01:33:42.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211941+0000", 2024-09-15T01:33:42.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021983546999999999 2024-09-15T01:33:42.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211947+0000", 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7329999999999997e-06 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211977+0000", 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9607e-05 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211982+0000", 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6290000000000001e-06 2024-09-15T01:33:42.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212053+0000", 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1890000000000005e-05 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212060+0000", 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3339999999999998e-06 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212087+0000", 2024-09-15T01:33:42.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7314e-05 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212098+0000", 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1141999999999999e-05 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212145+0000", 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699000000000001e-05 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.668 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:72 3.5s0 3:af8bec4d:::smithi17529310-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.439806731000004, 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.037268951000000002, 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72 2024-09-15T01:33:42.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.34e-07 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207319+0000", 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207326+0000", 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4860000000000002e-06 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207330+0000", 2024-09-15T01:33:42.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0690000000000003e-06 2024-09-15T01:33:42.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208687+0000", 2024-09-15T01:33:42.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001356966 2024-09-15T01:33:42.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:42.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208760+0000", 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2663000000000005e-05 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210380+0000", 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001620659 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210464+0000", 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3387999999999995e-05 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.244586+0000", 2024-09-15T01:33:42.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034121925999999997 2024-09-15T01:33:42.674 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.674 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.674 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.674 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.0s2 34/31 ECSubWrite(tid=15, reqid=client.4477.0:74, at_version=34'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:33:42.674 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.434695984000001, 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069361051000000007, 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212428+0000", 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3399999999999999e-07 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212444+0000", 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5681000000000001e-05 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281357+0000", 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068913881999999996 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281366+0000", 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9740000000000008e-06 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281415+0000", 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8498000000000002e-05 2024-09-15T01:33:42.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281421+0000", 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4590000000000003e-06 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281487+0000", 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5590000000000001e-05 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281500+0000", 2024-09-15T01:33:42.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3298e-05 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281575+0000", 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5247000000000006e-05 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281597+0000", 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1872999999999999e-05 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281788+0000", 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000191015 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 38/31 ECSubWrite(tid=17, reqid=client.4477.0:208, at_version=38'12, trim_to=0'0, roll_forward_to=36'8))", 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 62.425762085999999, 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.93429882900000005, 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221362+0000", 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9699999999999996e-07 2024-09-15T01:33:42.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221380+0000", 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.804e-05 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.154929+0000", 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.93354887600000003 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.154938+0000", 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6220000000000004e-06 2024-09-15T01:33:42.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155018+0000", 2024-09-15T01:33:42.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0136000000000007e-05 2024-09-15T01:33:42.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155027+0000", 2024-09-15T01:33:42.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8259999999999999e-06 2024-09-15T01:33:42.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155116+0000", 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8939999999999999e-05 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155127+0000", 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0981e-05 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155185+0000", 2024-09-15T01:33:42.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7828999999999998e-05 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155204+0000", 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8649999999999999e-05 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155660+0000", 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000456432 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.685 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 51/31 ECSubWrite(tid=56, reqid=osd.5.0:56, at_version=51'43, trim_to=0'0, roll_forward_to=50'41))", 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 49.570272352000003, 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.063539848999999995, 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076876+0000", 2024-09-15T01:33:42.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4834999999999999e-05 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139053+0000", 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.062177018000000001 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139067+0000", 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3842999999999999e-05 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139206+0000", 2024-09-15T01:33:42.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013896500000000001 2024-09-15T01:33:42.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139221+0000", 2024-09-15T01:33:42.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5193e-05 2024-09-15T01:33:42.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139472+0000", 2024-09-15T01:33:42.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025116 2024-09-15T01:33:42.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139504+0000", 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1282999999999998e-05 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139541+0000", 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7518999999999998e-05 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139557+0000", 2024-09-15T01:33:42.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5781e-05 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.140391+0000", 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00083395699999999997 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:505 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [copy-from ver 2] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 45.574085009000001, 2024-09-15T01:33:42.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15814687399999999, 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 505 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073040+0000", 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.792e-06 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073047+0000", 2024-09-15T01:33:42.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0169999999999999e-06 2024-09-15T01:33:42.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073064+0000", 2024-09-15T01:33:42.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.668e-05 2024-09-15T01:33:42.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073081+0000", 2024-09-15T01:33:42.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7000999999999999e-05 2024-09-15T01:33:42.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073158+0000", 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6779000000000005e-05 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073284+0000", 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000126802 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082517+0000", 2024-09-15T01:33:42.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0092324069999999998 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082822+0000", 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030544099999999999 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082921+0000", 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8760999999999999e-05 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082994+0000", 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2769999999999996e-05 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083217+0000", 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000223706 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231110+0000", 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.147892096 2024-09-15T01:33:42.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231185+0000", 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5622000000000002e-05 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 55/31 ECSubWriteReply(tid=49, last_complete=55'33, committed=1, applied=1))", 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 45.562841696, 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14718672399999999, 2024-09-15T01:33:42.698 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9599999999999998e-07 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084287+0000", 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.230928+0000", 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14664121799999999 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.230933+0000", 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4089999999999996e-06 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231018+0000", 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5589999999999999e-05 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231468+0000", 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045001999999999998 2024-09-15T01:33:42.701 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s4 56/54 ECSubWrite(tid=228, reqid=client.4477.0:520, at_version=56'33, trim_to=0'0, roll_forward_to=55'31))", 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 44.573121665000002, 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016830581000000001, 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:42.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:42.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:42.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7000000000000003e-07 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074027+0000", 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4845e-05 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074072+0000", 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4583000000000003e-05 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074080+0000", 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6699999999999994e-06 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074280+0000", 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020015699999999999 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074290+0000", 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0586e-05 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074562+0000", 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027202599999999997 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074573+0000", 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0974e-05 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074624+0000", 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0385e-05 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074786+0000", 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016278399999999999 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.090832+0000", 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016045700999999999 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 66/31 ECSubWrite(tid=130, reqid=osd.14.0:130, at_version=66'56, trim_to=0'0, roll_forward_to=60'52))", 2024-09-15T01:33:42.707 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 34.566864004999999, 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.31166652299999997, 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2000000000000001e-07 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080275+0000", 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.556e-05 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.390960+0000", 2024-09-15T01:33:42.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.31068479999999998 2024-09-15T01:33:42.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.390970+0000", 2024-09-15T01:33:42.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0223e-05 2024-09-15T01:33:42.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391016+0000", 2024-09-15T01:33:42.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5972e-05 2024-09-15T01:33:42.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391025+0000", 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8610000000000004e-06 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391232+0000", 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020709799999999999 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391264+0000", 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2347e-05 2024-09-15T01:33:42.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391319+0000", 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5096999999999998e-05 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391336+0000", 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6212e-05 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391926+0000", 2024-09-15T01:33:42.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00059013299999999996 2024-09-15T01:33:42.716 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.716 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.716 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.716 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:646 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [copy-from ver 59] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:42.716 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.077480+0000", 2024-09-15T01:33:42.716 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.569643711000001, 2024-09-15T01:33:42.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024635151000000001, 2024-09-15T01:33:42.716 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.716 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:42.716 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 646 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077480+0000", 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077480+0000", 2024-09-15T01:33:42.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077482+0000", 2024-09-15T01:33:42.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.967e-06 2024-09-15T01:33:42.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077494+0000", 2024-09-15T01:33:42.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2351000000000001e-05 2024-09-15T01:33:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077510+0000", 2024-09-15T01:33:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6461e-05 2024-09-15T01:33:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077528+0000", 2024-09-15T01:33:42.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7297000000000001e-05 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077583+0000", 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4877999999999999e-05 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op must wait for map", 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077588+0000", 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5809999999999996e-06 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.078844+0000", 2024-09-15T01:33:42.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001255693 2024-09-15T01:33:42.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.078981+0000", 2024-09-15T01:33:42.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013696499999999999 2024-09-15T01:33:42.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097046+0000", 2024-09-15T01:33:42.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018065193 2024-09-15T01:33:42.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097686+0000", 2024-09-15T01:33:42.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00063973600000000002 2024-09-15T01:33:42.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097704+0000", 2024-09-15T01:33:42.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7934000000000001e-05 2024-09-15T01:33:42.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098011+0000", 2024-09-15T01:33:42.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030703799999999998 2024-09-15T01:33:42.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098161+0000", 2024-09-15T01:33:42.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000150233 2024-09-15T01:33:42.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:42.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.102063+0000", 2024-09-15T01:33:42.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003902109 2024-09-15T01:33:42.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.102115+0000", 2024-09-15T01:33:42.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1715000000000002e-05 2024-09-15T01:33:42.725 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.725 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.725 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.726 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:647 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:42.726 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.077541+0000", 2024-09-15T01:33:42.726 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.569582564000001, 2024-09-15T01:33:42.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020616064, 2024-09-15T01:33:42.726 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.726 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:42.726 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 647 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077541+0000", 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077541+0000", 2024-09-15T01:33:42.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077542+0000", 2024-09-15T01:33:42.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2099999999999995e-07 2024-09-15T01:33:42.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077545+0000", 2024-09-15T01:33:42.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5360000000000001e-06 2024-09-15T01:33:42.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077555+0000", 2024-09-15T01:33:42.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0193e-05 2024-09-15T01:33:42.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077561+0000", 2024-09-15T01:33:42.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5679999999999999e-06 2024-09-15T01:33:42.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077630+0000", 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9648999999999997e-05 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077635+0000", 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0050000000000004e-06 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.079228+0000", 2024-09-15T01:33:42.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015930009999999999 2024-09-15T01:33:42.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:42.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.079271+0000", 2024-09-15T01:33:42.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2588e-05 2024-09-15T01:33:42.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097903+0000", 2024-09-15T01:33:42.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018632124999999999 2024-09-15T01:33:42.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098048+0000", 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000144979 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098157+0000", 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010859899999999999 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:676 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [setxattr _header (56) in=7b] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.376552781000001, 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054773589999999997, 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 676 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.733 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:42.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:42.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1199999999999999e-07 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270573+0000", 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7409999999999999e-06 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270576+0000", 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0580000000000002e-06 2024-09-15T01:33:42.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270579+0000", 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0630000000000002e-06 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273029+0000", 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024502119999999998 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273139+0000", 2024-09-15T01:33:42.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110196 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273456+0000", 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031676000000000002 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273543+0000", 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7276999999999998e-05 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273550+0000", 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2119999999999997e-06 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273583+0000", 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2625e-05 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273919+0000", 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033556900000000003 2024-09-15T01:33:42.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325294+0000", 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051375250999999997 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325344+0000", 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0314e-05 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.740 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:677 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:42.740 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:42.740 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.376531487000001, 2024-09-15T01:33:42.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.058358134999999998, 2024-09-15T01:33:42.740 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.740 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:42.740 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.740 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.740 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.740 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 677 2024-09-15T01:33:42.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.740 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.65e-07 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270594+0000", 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4219999999999999e-06 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270598+0000", 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9439999999999998e-06 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270600+0000", 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.243e-06 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.274286+0000", 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036864089999999999 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.274337+0000", 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0677999999999998e-05 2024-09-15T01:33:42.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325802+0000", 2024-09-15T01:33:42.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051465576999999998 2024-09-15T01:33:42.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325904+0000", 2024-09-15T01:33:42.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010192 2024-09-15T01:33:42.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328950+0000", 2024-09-15T01:33:42.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030455769999999998 2024-09-15T01:33:42.745 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.745 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.745 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.745 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 71/31 ECSubWriteReply(tid=63, last_complete=71'46, committed=1, applied=1))", 2024-09-15T01:33:42.745 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.273846+0000", 2024-09-15T01:33:42.745 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.373276937, 2024-09-15T01:33:42.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051825606000000003, 2024-09-15T01:33:42.745 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273846+0000", 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273846+0000", 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273847+0000", 2024-09-15T01:33:42.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5199999999999997e-07 2024-09-15T01:33:42.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273850+0000", 2024-09-15T01:33:42.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1690000000000001e-06 2024-09-15T01:33:42.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325190+0000", 2024-09-15T01:33:42.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051340157999999997 2024-09-15T01:33:42.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325198+0000", 2024-09-15T01:33:42.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4839999999999998e-06 2024-09-15T01:33:42.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325235+0000", 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7156000000000001e-05 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325672+0000", 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00043708700000000001 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 71/31 ECSubWriteReply(tid=64, last_complete=71'46, committed=1, applied=1))", 2024-09-15T01:33:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.329874+0000", 2024-09-15T01:33:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.317249307000001, 2024-09-15T01:33:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.074638840999999997, 2024-09-15T01:33:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329874+0000", 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329874+0000", 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329875+0000", 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2799999999999996e-07 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329878+0000", 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7579999999999998e-06 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404471+0000", 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.074592695000000001 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404475+0000", 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8650000000000003e-06 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404497+0000", 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2458000000000001e-05 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404513+0000", 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5537e-05 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:714 3.5s0 3:a0a9f8ce:::smithi17529310-35:head [delete] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+full_force+supports_pool_eio e71)", 2024-09-15T01:33:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.401661+0000", 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.245462455999998, 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044420634000000001, 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 714 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401661+0000", 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401661+0000", 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401663+0000", 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401670+0000", 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3549999999999999e-06 2024-09-15T01:33:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401687+0000", 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7388e-05 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401695+0000", 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6310000000000003e-06 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401732+0000", 2024-09-15T01:33:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7415e-05 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401813+0000", 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0117999999999994e-05 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402335+0000", 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052289800000000005 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402584+0000", 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024863700000000001 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402594+0000", 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7890000000000006e-06 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402634+0000", 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9984000000000002e-05 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402647+0000", 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3105000000000001e-05 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446020+0000", 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.043373215 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446081+0000", 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1248999999999996e-05 2024-09-15T01:33:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 71/31 ECSubWriteReply(tid=66, last_complete=71'48, committed=1, applied=1))", 2024-09-15T01:33:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.242668600000002, 2024-09-15T01:33:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041833197000000003, 2024-09-15T01:33:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2099999999999997e-07 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404458+0000", 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.976e-06 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.445921+0000", 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041463284000000003 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.445925+0000", 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0260000000000001e-06 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.445975+0000", 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9424e-05 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446288+0000", 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031296600000000003 2024-09-15T01:33:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s4 86/54 ECSubWrite(tid=305, reqid=client.4477.0:848, at_version=86'56, trim_to=0'0, roll_forward_to=85'52))", 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.135651+0000", 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.511472373, 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01654541, 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135651+0000", 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135651+0000", 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135651+0000", 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2000000000000001e-07 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135665+0000", 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3978000000000001e-05 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135726+0000", 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1020000000000002e-05 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135730+0000", 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.929e-06 2024-09-15T01:33:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135946+0000", 2024-09-15T01:33:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021585699999999999 2024-09-15T01:33:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135955+0000", 2024-09-15T01:33:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0929999999999992e-06 2024-09-15T01:33:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136046+0000", 2024-09-15T01:33:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0948999999999999e-05 2024-09-15T01:33:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136096+0000", 2024-09-15T01:33:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0049999999999997e-05 2024-09-15T01:33:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136268+0000", 2024-09-15T01:33:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000172127 2024-09-15T01:33:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136324+0000", 2024-09-15T01:33:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5379e-05 2024-09-15T01:33:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.152196+0000", 2024-09-15T01:33:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015872807999999999 2024-09-15T01:33:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:42.772 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:42.773 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:42.773 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:42.774 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:42.774 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:33:42.775 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:42.776 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:42.772+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:42.874 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:42.872+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:42.916 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:42.916 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:42.925 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:33:42.975 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:42.973+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:43.025 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:43.025 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:43.034 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:33:43.075 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-15T01:33:43.074+0000 7fd0f6df9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28501) UID: 0 2024-09-15T01:33:43.122 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-15T01:33:43.122 INFO:teuthology.orchestra.run.smithi042.stderr:osd set-full-ratio 0.95 2024-09-15T01:33:43.130 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 29 v 239 (ObjNum 123 snap 20 seq_num 123) dirty exists 2024-09-15T01:33:43.130 INFO:tasks.rados.rados.0.smithi175.stdout:386: expect (ObjNum 139 snap 29 seq_num 139) 2024-09-15T01:33:43.149 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:43.149 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:43.162 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:33:43.176 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:43.174+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:43.204 INFO:tasks.rados.rados.0.smithi175.stdout:385: done (2 left) 2024-09-15T01:33:43.204 INFO:tasks.rados.rados.0.smithi175.stdout:386: done (1 left) 2024-09-15T01:33:43.204 INFO:tasks.rados.rados.0.smithi175.stdout:387: done (0 left) 2024-09-15T01:33:43.204 INFO:tasks.rados.rados.0.smithi175.stdout:388: read oid 1 snap -1 2024-09-15T01:33:43.204 INFO:tasks.rados.rados.0.smithi175.stdout:388: expect deleted 2024-09-15T01:33:43.204 INFO:tasks.rados.rados.0.smithi175.stdout:389: copy_from oid 5 from oid 50 current snap is 34 2024-09-15T01:33:43.205 INFO:tasks.rados.rados.0.smithi175.stdout:390: rollback oid 24 current snap is 34 2024-09-15T01:33:43.205 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 24 to 31 2024-09-15T01:33:43.205 INFO:tasks.rados.rados.0.smithi175.stdout:391: rmattr oid 17 current snap is 34 2024-09-15T01:33:43.205 INFO:tasks.rados.rados.0.smithi175.stdout:391: done (3 left) 2024-09-15T01:33:43.205 INFO:tasks.rados.rados.0.smithi175.stdout:392: read oid 39 snap -1 2024-09-15T01:33:43.205 INFO:tasks.rados.rados.0.smithi175.stdout:392: expect (ObjNum 139 snap 29 seq_num 139) 2024-09-15T01:33:43.205 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 28 current snap is 34 2024-09-15T01:33:43.205 INFO:tasks.rados.rados.0.smithi175.stdout:393: seq_num 156 ranges {1703936=229376} 2024-09-15T01:33:43.214 INFO:tasks.rados.rados.0.smithi175.stdout:393: writing smithi17529310-28 from 1703936 to 1933312 tid 1 2024-09-15T01:33:43.214 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 38 current snap is 34 2024-09-15T01:33:43.214 INFO:tasks.rados.rados.0.smithi175.stdout:394: seq_num 157 ranges {98304=245760} 2024-09-15T01:33:43.216 INFO:tasks.rados.rados.0.smithi175.stdout:394: writing smithi17529310-38 from 98304 to 344064 tid 1 2024-09-15T01:33:43.217 INFO:tasks.rados.rados.0.smithi175.stdout:395: rollback oid 16 current snap is 34 2024-09-15T01:33:43.217 INFO:tasks.rados.rados.0.smithi175.stdout:rollback oid 16 to 26 2024-09-15T01:33:43.217 INFO:tasks.rados.rados.0.smithi175.stdout:390: finishing rollback tid 1 to smithi17529310-24 2024-09-15T01:33:43.217 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 24 v 0 (ObjNum 129 snap 0 seq_num 1648494024) dirty dne 2024-09-15T01:33:43.217 INFO:tasks.rados.rados.0.smithi175.stdout:392: expect (ObjNum 139 snap 29 seq_num 139) 2024-09-15T01:33:43.277 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:43.275+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:43.287 INFO:tasks.rados.rados.0.smithi175.stdout:393: finishing write tid 1 to smithi17529310-28 2024-09-15T01:33:43.287 INFO:tasks.rados.rados.0.smithi175.stdout:393: finishing write tid 2 to smithi17529310-28 2024-09-15T01:33:43.287 INFO:tasks.rados.rados.0.smithi175.stdout:395: finishing rollback tid 0 to smithi17529310-16 2024-09-15T01:33:43.287 INFO:tasks.rados.rados.0.smithi175.stdout:388: done (6 left) 2024-09-15T01:33:43.287 INFO:tasks.rados.rados.0.smithi175.stdout:390: done (5 left) 2024-09-15T01:33:43.287 INFO:tasks.rados.rados.0.smithi175.stdout:392: done (4 left) 2024-09-15T01:33:43.287 INFO:tasks.rados.rados.0.smithi175.stdout:396: copy_from oid 33 from oid 35 current snap is 34 2024-09-15T01:33:43.287 INFO:tasks.rados.rados.0.smithi175.stdout:394: finishing write tid 1 to smithi17529310-38 2024-09-15T01:33:43.287 INFO:tasks.rados.rados.0.smithi175.stdout:393: finishing write tid 3 to smithi17529310-28 2024-09-15T01:33:43.288 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 28 v 138 (ObjNum 156 snap 34 seq_num 156) dirty exists 2024-09-15T01:33:43.288 INFO:tasks.rados.rados.0.smithi175.stdout:393: left oid 28 (ObjNum 156 snap 34 seq_num 156) 2024-09-15T01:33:43.288 INFO:tasks.rados.rados.0.smithi175.stdout:395: finishing rollback tid 1 to smithi17529310-16 2024-09-15T01:33:43.288 INFO:tasks.rados.rados.0.smithi175.stdout:393: done (4 left) 2024-09-15T01:33:43.288 INFO:tasks.rados.rados.0.smithi175.stdout:append oid 11 current snap is 34 2024-09-15T01:33:43.288 INFO:tasks.rados.rados.0.smithi175.stdout:397: seq_num 158 ranges {0=262144} 2024-09-15T01:33:43.288 INFO:tasks.rados.rados.0.smithi175.stdout:397: writing smithi17529310-11 from 0 to 262144 tid 1 2024-09-15T01:33:43.288 INFO:tasks.rados.rados.0.smithi175.stdout:398: snap_remove snap 32 2024-09-15T01:33:43.305 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:43.306 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:43.314 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:33:43.378 INFO:tasks.ceph.osd.11.smithi175.stderr:2024-09-15T01:33:43.376+0000 7f56fe3f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 28831) UID: 0 2024-09-15T01:33:43.455 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:43.455 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:43.464 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:33:43.507 INFO:tasks.ceph.osd.8.smithi042.stderr:2024-09-15T01:33:43.506+0000 7f04479f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 28840) UID: 0 2024-09-15T01:33:43.564 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:43.565 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:43.573 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:33:43.579 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:43.578+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:43.681 INFO:tasks.ceph.osd.2.smithi152.stderr:2024-09-15T01:33:43.679+0000 7f2fe43f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 28723) UID: 0 2024-09-15T01:33:43.715 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:43.716 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:43.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:33:43.782 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-15T01:33:43.780+0000 7f17d73f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28498) UID: 0 2024-09-15T01:33:43.825 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:43.825 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:43.834 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:33:43.882 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-09-15T01:33:43.881+0000 7ff44fbf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28722) UID: 0 2024-09-15T01:33:43.936 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:43.936 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:43.944 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-15T01:33:44.002 INFO:tasks.ceph.osd.13.smithi115.stderr:2024-09-15T01:33:43.982+0000 7f6bd9ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 28506) UID: 0 2024-09-15T01:33:44.061 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:44.061 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:44.068 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-15T01:33:44.088 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:44.082+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:44.128 INFO:tasks.rados.rados.0.smithi175.stdout:398: done (5 left) 2024-09-15T01:33:44.128 INFO:tasks.rados.rados.0.smithi175.stdout:399: copy_from oid 2 from oid 8 current snap is 34 2024-09-15T01:33:44.129 INFO:tasks.rados.rados.0.smithi175.stdout:397: finishing write tid 1 to smithi17529310-11 2024-09-15T01:33:44.129 INFO:tasks.rados.rados.0.smithi175.stdout:395: finishing rollback tid 2 to smithi17529310-16 2024-09-15T01:33:44.129 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 16 v 141 (ObjNum 134 snap 24 seq_num 134) dirty exists 2024-09-15T01:33:44.129 INFO:tasks.rados.rados.0.smithi175.stdout:397: finishing write tid 2 to smithi17529310-11 2024-09-15T01:33:44.129 INFO:tasks.rados.rados.0.smithi175.stdout:396: finishing copy_from to smithi17529310-33 2024-09-15T01:33:44.129 INFO:tasks.rados.rados.0.smithi175.stdout:396: got expected ENOENT (src dne) 2024-09-15T01:33:44.129 INFO:tasks.rados.rados.0.smithi175.stdout:396: finishing copy_from racing read to smithi17529310-33 2024-09-15T01:33:44.129 INFO:tasks.rados.rados.0.smithi175.stdout:397: finishing write tid 3 to smithi17529310-11 2024-09-15T01:33:44.129 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 11 v 40 (ObjNum 158 snap 34 seq_num 158) dirty exists 2024-09-15T01:33:44.129 INFO:tasks.rados.rados.0.smithi175.stdout:397: left oid 11 (ObjNum 158 snap 34 seq_num 158) 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout:389: finishing copy_from racing read to smithi17529310-5 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout:389: finishing copy_from to smithi17529310-5 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 5 v 45 (ObjNum 149 snap 31 seq_num 149) dirty exists 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout:394: finishing write tid 2 to smithi17529310-38 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout:394: finishing write tid 3 to smithi17529310-38 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 38 v 254 (ObjNum 157 snap 34 seq_num 157) dirty exists 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout:394: left oid 38 (ObjNum 157 snap 34 seq_num 157) 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout:389: done (5 left) 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout:394: done (4 left) 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout:395: done (3 left) 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout:396: done (2 left) 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout:397: done (1 left) 2024-09-15T01:33:44.130 INFO:tasks.rados.rados.0.smithi175.stdout: waiting on 1 2024-09-15T01:33:44.134 INFO:tasks.rados.rados.0.smithi175.stdout:399: finishing copy_from racing read to smithi17529310-2 2024-09-15T01:33:44.141 INFO:tasks.rados.rados.0.smithi175.stdout:399: finishing copy_from to smithi17529310-2 2024-09-15T01:33:44.141 INFO:tasks.rados.rados.0.smithi175.stdout:update_object_version oid 2 v 72 (ObjNum 144 snap 30 seq_num 144) dirty exists 2024-09-15T01:33:44.141 INFO:tasks.rados.rados.0.smithi175.stdout:399: done (0 left) 2024-09-15T01:33:44.142 INFO:tasks.rados.rados.0.smithi175.stderr:0 errors. 2024-09-15T01:33:44.143 INFO:tasks.rados.rados.0.smithi175.stderr: 2024-09-15T01:33:44.145 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:33:44.185 INFO:tasks.ceph.osd.9.smithi115.stderr:2024-09-15T01:33:44.183+0000 7ff0fb1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 28504) UID: 0 2024-09-15T01:33:44.191 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:44.191 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:44.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-15T01:33:44.286 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:44.284+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:44.304 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:44.304 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:44.313 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:33:44.386 INFO:tasks.ceph.osd.10.smithi152.stderr:2024-09-15T01:33:44.385+0000 7f6304bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 28727) UID: 0 2024-09-15T01:33:44.424 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:44.424 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:33:44.427 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":102,"fsid":"d4803586-ed44-4154-8ee1-6b95ec3e17e7","created":"2024-09-15T01:32:07.285177+0000","modified":"2024-09-15T01:33:44.118544+0000","last_up_change":"2024-09-15T01:32:14.443323+0000","last_in_change":"2024-09-15T01:33:08.068993+0000","flags":"noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5801984,"flags_set":["noscrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:32:15.118634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:32:21.824131+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-09-15T01:32:32.183257+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":7,"min_size":5,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":66,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"shecprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":16384,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"bec41583-a0a4-4712-a9ec-eec1e88fb60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":28861},{"type":"v1","addr":"172.21.15.42:6809","nonce":28861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":28861},{"type":"v1","addr":"172.21.15.42:6811","nonce":28861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":28861},{"type":"v1","addr":"172.21.15.42:6815","nonce":28861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":28861},{"type":"v1","addr":"172.21.15.42:6813","nonce":28861}]},"public_addr":"172.21.15.42:6809/28861","cluster_addr":"172.21.15.42:6811/28861","heartbeat_back_addr":"172.21.15.42:6815/28861","heartbeat_front_addr":"172.21.15.42:6813/28861","state":["exists","up"]},{"osd":1,"uuid":"2cca66be-8685-4a38-abf8-c864cd5434da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":28508},{"type":"v1","addr":"172.21.15.115:6809","nonce":28508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":28508},{"type":"v1","addr":"172.21.15.115:6811","nonce":28508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6817","nonce":28508},{"type":"v1","addr":"172.21.15.115:6819","nonce":28508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6813","nonce":28508},{"type":"v1","addr":"172.21.15.115:6815","nonce":28508}]},"public_addr":"172.21.15.115:6809/28508","cluster_addr":"172.21.15.115:6811/28508","heartbeat_back_addr":"172.21.15.115:6819/28508","heartbeat_front_addr":"172.21.15.115:6815/28508","state":["exists","up"]},{"osd":2,"uuid":"b8756aac-3a3e-41ba-a6ef-497063cb44df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":28732},{"type":"v1","addr":"172.21.15.152:6809","nonce":28732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":28732},{"type":"v1","addr":"172.21.15.152:6811","nonce":28732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":28732},{"type":"v1","addr":"172.21.15.152:6816","nonce":28732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":28732},{"type":"v1","addr":"172.21.15.152:6813","nonce":28732}]},"public_addr":"172.21.15.152:6809/28732","cluster_addr":"172.21.15.152:6811/28732","heartbeat_back_addr":"172.21.15.152:6816/28732","heartbeat_front_addr":"172.21.15.152:6813/28732","state":["exists","up"]},{"osd":3,"uuid":"554875f5-74c0-4b58-80ee-6ba93748f104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":28835},{"type":"v1","addr":"172.21.15.175:6803","nonce":28835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":28835},{"type":"v1","addr":"172.21.15.175:6805","nonce":28835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":28835},{"type":"v1","addr":"172.21.15.175:6809","nonce":28835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":28835},{"type":"v1","addr":"172.21.15.175:6807","nonce":28835}]},"public_addr":"172.21.15.175:6803/28835","cluster_addr":"172.21.15.175:6805/28835","heartbeat_back_addr":"172.21.15.175:6809/28835","heartbeat_front_addr":"172.21.15.175:6807/28835","state":["exists","up"]},{"osd":4,"uuid":"3fe2c05a-4c22-4dd7-a8c6-282155c82eb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":28860},{"type":"v1","addr":"172.21.15.42:6801","nonce":28860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":28860},{"type":"v1","addr":"172.21.15.42:6803","nonce":28860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":28860},{"type":"v1","addr":"172.21.15.42:6807","nonce":28860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":28860},{"type":"v1","addr":"172.21.15.42:6805","nonce":28860}]},"public_addr":"172.21.15.42:6801/28860","cluster_addr":"172.21.15.42:6803/28860","heartbeat_back_addr":"172.21.15.42:6807/28860","heartbeat_front_addr":"172.21.15.42:6805/28860","state":["exists","up"]},{"osd":5,"uuid":"a301d1ce-3e31-4782-ade3-f9c0f1e6a95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":28509},{"type":"v1","addr":"172.21.15.115:6801","nonce":28509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":28509},{"type":"v1","addr":"172.21.15.115:6803","nonce":28509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":28509},{"type":"v1","addr":"172.21.15.115:6807","nonce":28509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":28509},{"type":"v1","addr":"172.21.15.115:6805","nonce":28509}]},"public_addr":"172.21.15.115:6801/28509","cluster_addr":"172.21.15.115:6803/28509","heartbeat_back_addr":"172.21.15.115:6807/28509","heartbeat_front_addr":"172.21.15.115:6805/28509","state":["exists","up"]},{"osd":6,"uuid":"c7d180be-21c5-4674-a9f2-10a1c7794871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":28731},{"type":"v1","addr":"172.21.15.152:6801","nonce":28731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":28731},{"type":"v1","addr":"172.21.15.152:6803","nonce":28731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":28731},{"type":"v1","addr":"172.21.15.152:6807","nonce":28731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":28731},{"type":"v1","addr":"172.21.15.152:6805","nonce":28731}]},"public_addr":"172.21.15.152:6801/28731","cluster_addr":"172.21.15.152:6803/28731","heartbeat_back_addr":"172.21.15.152:6807/28731","heartbeat_front_addr":"172.21.15.152:6805/28731","state":["exists","up"]},{"osd":7,"uuid":"4d2429ea-772f-4fb9-a421-240f1399f16c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":28836},{"type":"v1","addr":"172.21.15.175:6811","nonce":28836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":28836},{"type":"v1","addr":"172.21.15.175:6813","nonce":28836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":28836},{"type":"v1","addr":"172.21.15.175:6817","nonce":28836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":28836},{"type":"v1","addr":"172.21.15.175:6815","nonce":28836}]},"public_addr":"172.21.15.175:6811/28836","cluster_addr":"172.21.15.175:6813/28836","heartbeat_back_addr":"172.21.15.175:6817/28836","heartbeat_front_addr":"172.21.15.175:6815/28836","state":["exists","up"]},{"osd":8,"uuid":"de7389ab-a29e-4177-9e94-484fc6a94102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":28862},{"type":"v1","addr":"172.21.15.42:6817","nonce":28862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":28862},{"type":"v1","addr":"172.21.15.42:6819","nonce":28862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":28862},{"type":"v1","addr":"172.21.15.42:6823","nonce":28862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":28862},{"type":"v1","addr":"172.21.15.42:6821","nonce":28862}]},"public_addr":"172.21.15.42:6817/28862","cluster_addr":"172.21.15.42:6819/28862","heartbeat_back_addr":"172.21.15.42:6823/28862","heartbeat_front_addr":"172.21.15.42:6821/28862","state":["exists","up"]},{"osd":9,"uuid":"4b4b44da-97d6-41f2-8fc6-57680b10aeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":28510},{"type":"v1","addr":"172.21.15.115:6814","nonce":28510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":28510},{"type":"v1","addr":"172.21.15.115:6818","nonce":28510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":28510},{"type":"v1","addr":"172.21.15.115:6823","nonce":28510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":28510},{"type":"v1","addr":"172.21.15.115:6821","nonce":28510}]},"public_addr":"172.21.15.115:6814/28510","cluster_addr":"172.21.15.115:6818/28510","heartbeat_back_addr":"172.21.15.115:6823/28510","heartbeat_front_addr":"172.21.15.115:6821/28510","state":["exists","up"]},{"osd":10,"uuid":"5f285a07-9057-4575-b911-b3977acd397c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":28733},{"type":"v1","addr":"172.21.15.152:6817","nonce":28733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":28733},{"type":"v1","addr":"172.21.15.152:6819","nonce":28733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":28733},{"type":"v1","addr":"172.21.15.152:6824","nonce":28733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":28733},{"type":"v1","addr":"172.21.15.152:6821","nonce":28733}]},"public_addr":"172.21.15.152:6817/28733","cluster_addr":"172.21.15.152:6819/28733","heartbeat_back_addr":"172.21.15.152:6824/28733","heartbeat_front_addr":"172.21.15.152:6821/28733","state":["exists","up"]},{"osd":11,"uuid":"dacc0f3f-a598-4055-92f4-d41102ceca83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":28837},{"type":"v1","addr":"172.21.15.175:6827","nonce":28837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":28837},{"type":"v1","addr":"172.21.15.175:6829","nonce":28837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":28837},{"type":"v1","addr":"172.21.15.175:6833","nonce":28837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":28837},{"type":"v1","addr":"172.21.15.175:6831","nonce":28837}]},"public_addr":"172.21.15.175:6827/28837","cluster_addr":"172.21.15.175:6829/28837","heartbeat_back_addr":"172.21.15.175:6833/28837","heartbeat_front_addr":"172.21.15.175:6831/28837","state":["exists","up"]},{"osd":12,"uuid":"d218197b-d0db-410c-abff-8b849a21a555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":28866},{"type":"v1","addr":"172.21.15.42:6825","nonce":28866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":28866},{"type":"v1","addr":"172.21.15.42:6827","nonce":28866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":28866},{"type":"v1","addr":"172.21.15.42:6831","nonce":28866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":28866},{"type":"v1","addr":"172.21.15.42:6829","nonce":28866}]},"public_addr":"172.21.15.42:6825/28866","cluster_addr":"172.21.15.42:6827/28866","heartbeat_back_addr":"172.21.15.42:6831/28866","heartbeat_front_addr":"172.21.15.42:6829/28866","state":["exists","up"]},{"osd":13,"uuid":"c9bf5faf-f2ee-482e-b903-28c7718e2063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":28511},{"type":"v1","addr":"172.21.15.115:6825","nonce":28511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":28511},{"type":"v1","addr":"172.21.15.115:6827","nonce":28511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":28511},{"type":"v1","addr":"172.21.15.115:6831","nonce":28511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":28511},{"type":"v1","addr":"172.21.15.115:6829","nonce":28511}]},"public_addr":"172.21.15.115:6825/28511","cluster_addr":"172.21.15.115:6827/28511","heartbeat_back_addr":"172.21.15.115:6831/28511","heartbeat_front_addr":"172.21.15.115:6829/28511","state":["exists","up"]},{"osd":14,"uuid":"cc3f19cf-5a97-440b-a953-7727e6ed6a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6823","nonce":28734},{"type":"v1","addr":"172.21.15.152:6825","nonce":28734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":28734},{"type":"v1","addr":"172.21.15.152:6827","nonce":28734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":28734},{"type":"v1","addr":"172.21.15.152:6831","nonce":28734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":28734},{"type":"v1","addr":"172.21.15.152:6829","nonce":28734}]},"public_addr":"172.21.15.152:6825/28734","cluster_addr":"172.21.15.152:6827/28734","heartbeat_back_addr":"172.21.15.152:6831/28734","heartbeat_front_addr":"172.21.15.152:6829/28734","state":["exists","up"]},{"osd":15,"uuid":"51290c23-e86d-4e05-9ac8-ca53d6a1eb80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":28838},{"type":"v1","addr":"172.21.15.175:6819","nonce":28838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":28838},{"type":"v1","addr":"172.21.15.175:6821","nonce":28838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":28838},{"type":"v1","addr":"172.21.15.175:6825","nonce":28838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":28838},{"type":"v1","addr":"172.21.15.175:6823","nonce":28838}]},"public_addr":"172.21.15.175:6819/28838","cluster_addr":"172.21.15.175:6821/28838","heartbeat_back_addr":"172.21.15.175:6825/28838","heartbeat_front_addr":"172.21.15.175:6823/28838","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:15.503649+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:25.772185+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:32.378075+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:25.443004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:23.634325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.175620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.608100+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.434930+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:20.501276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:13.348208+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.326775+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.748721+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.563588+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:27.752979+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:19.554554+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:32.885128+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[10,8]},{"pgid":"3.9","osds":[11,3,6,0,13,5,9]}],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":25,"length":1},{"begin":31,"length":1},{"begin":41,"length":1},{"begin":44,"length":1},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":58,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":58,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:33:44.434 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:33:44.440 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:33:44.488 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-15T01:33:44.486+0000 7f957e5f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 28834) UID: 0 2024-09-15T01:33:44.531 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:44.532 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:44.539 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:33:44.588 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-09-15T01:33:44.586+0000 7f9772ff9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28835) UID: 0 2024-09-15T01:33:44.636 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:44.637 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:44.645 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:33:44.689 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:44.687+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:44.724 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:33:44.725 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:33:44.735 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-15T01:33:44.123367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117654067,"num_objects":94,"num_object_clones":47,"num_object_copies":638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":3,"num_read":487,"num_read_kb":216862,"num_write":977,"num_write_kb":169778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":44957696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1554,"ondisk_log_size":1554,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":13171680,"kb_used_data":1485707776,"kb_used_omap":43821,"kb_used_meta":0,"kb_avail":1485702176,"statfs":{"total":1534846828544,"available":1521359028224,"internally_reserved":5734400,"allocated":1521364762624,"data_stored":1521364762624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44873051,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":577,"commit_latency_ns":0,"apply_latency_ns":577000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5963776,"num_objects":-9,"num_object_clones":-8,"num_object_copies":-63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9,"num_whiteouts":-1,"num_read":9,"num_read_kb":12245,"num_write":2,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016119"},"pg_stats":[{"pgid":"3.f","version":"97'229","reported_seq":423,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T01:33:39.110790+0000","last_change":"2024-09-15T01:33:39.111358+0000","last_active":"2024-09-15T01:33:39.110790+0000","last_peered":"2024-09-15T01:33:39.110790+0000","last_clean":"2024-09-15T01:33:39.110790+0000","last_became_active":"2024-09-15T01:32:34.487273+0000","last_became_peered":"2024-09-15T01:32:34.487273+0000","last_unstale":"2024-09-15T01:33:39.110790+0000","last_undegraded":"2024-09-15T01:33:39.110790+0000","last_fullsized":"2024-09-15T01:33:39.110790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:42.230488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046357050000000004,"stat_sum":{"num_bytes":15400960,"num_objects":13,"num_object_clones":7,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":57,"num_read_kb":47960,"num_write":135,"num_write_kb":27616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":239,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T01:33:40.113401+0000","last_change":"2024-09-15T01:33:40.113843+0000","last_active":"2024-09-15T01:33:40.113401+0000","last_peered":"2024-09-15T01:33:40.113401+0000","last_clean":"2024-09-15T01:33:40.113401+0000","last_became_active":"2024-09-15T01:33:09.327259+0000","last_became_peered":"2024-09-15T01:33:09.327259+0000","last_unstale":"2024-09-15T01:33:40.113401+0000","last_undegraded":"2024-09-15T01:33:40.113401+0000","last_fullsized":"2024-09-15T01:33:40.113401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:57.127309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057233700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,1,11,8,6,12],"acting":[2,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.d","version":"98'123","reported_seq":325,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T01:33:40.133341+0000","last_change":"2024-09-15T01:33:40.134226+0000","last_active":"2024-09-15T01:33:40.133341+0000","last_peered":"2024-09-15T01:33:40.133341+0000","last_clean":"2024-09-15T01:33:40.133341+0000","last_became_active":"2024-09-15T01:32:34.487040+0000","last_became_peered":"2024-09-15T01:32:34.487040+0000","last_unstale":"2024-09-15T01:33:40.133341+0000","last_undegraded":"2024-09-15T01:33:40.133341+0000","last_fullsized":"2024-09-15T01:33:40.133341+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.025815630999999999,"stat_sum":{"num_bytes":12353536,"num_objects":9,"num_object_clones":5,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":39,"num_read_kb":25268,"num_write":58,"num_write_kb":13120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.c","version":"96'131","reported_seq":328,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T01:33:39.106769+0000","last_change":"2024-09-15T01:33:39.107063+0000","last_active":"2024-09-15T01:33:39.106769+0000","last_peered":"2024-09-15T01:33:39.106769+0000","last_clean":"2024-09-15T01:33:39.106769+0000","last_became_active":"2024-09-15T01:32:34.487072+0000","last_became_peered":"2024-09-15T01:32:34.487072+0000","last_unstale":"2024-09-15T01:33:39.106769+0000","last_undegraded":"2024-09-15T01:33:39.106769+0000","last_fullsized":"2024-09-15T01:33:39.106769+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:40.186197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050591199999999996,"stat_sum":{"num_bytes":13533184,"num_objects":8,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":31,"num_read_kb":28816,"num_write":70,"num_write_kb":16304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.b","version":"98'51","reported_seq":321,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T01:33:40.119533+0000","last_change":"2024-09-15T01:33:40.120191+0000","last_active":"2024-09-15T01:33:40.119533+0000","last_peered":"2024-09-15T01:33:40.119533+0000","last_clean":"2024-09-15T01:33:40.119533+0000","last_became_active":"2024-09-15T01:33:12.080816+0000","last_became_peered":"2024-09-15T01:33:12.080816+0000","last_unstale":"2024-09-15T01:33:40.119533+0000","last_undegraded":"2024-09-15T01:33:40.119533+0000","last_fullsized":"2024-09-15T01:33:40.119533+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:58.058984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089523009999999993,"stat_sum":{"num_bytes":4702208,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":18,"num_write_kb":6736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":3538944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.a","version":"98'59","reported_seq":368,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T01:33:40.114399+0000","last_change":"2024-09-15T01:33:40.115102+0000","last_active":"2024-09-15T01:33:40.114399+0000","last_peered":"2024-09-15T01:33:40.114399+0000","last_clean":"2024-09-15T01:33:40.114399+0000","last_became_active":"2024-09-15T01:33:12.077513+0000","last_became_peered":"2024-09-15T01:33:12.077513+0000","last_unstale":"2024-09-15T01:33:40.114399+0000","last_undegraded":"2024-09-15T01:33:40.114399+0000","last_fullsized":"2024-09-15T01:33:40.114399+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:58.853412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048606650000000001,"stat_sum":{"num_bytes":1572864,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":18,"num_write_kb":5584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":7471104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.9","version":"97'84","reported_seq":122,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T01:33:39.111222+0000","last_change":"2024-09-15T01:33:39.111892+0000","last_active":"2024-09-15T01:33:39.111222+0000","last_peered":"2024-09-15T01:33:39.111222+0000","last_clean":"2024-09-15T01:33:39.111222+0000","last_became_active":"2024-09-15T01:33:19.083379+0000","last_became_peered":"2024-09-15T01:33:19.083379+0000","last_unstale":"2024-09-15T01:33:39.111222+0000","last_undegraded":"2024-09-15T01:33:39.111222+0000","last_fullsized":"2024-09-15T01:33:39.111222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:58.063827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053555870000000002,"stat_sum":{"num_bytes":1294336,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":2988,"num_write":33,"num_write_kb":6832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6,0,13,5,9],"acting":[11,3,6,0,13,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.8","version":"96'123","reported_seq":319,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T01:33:40.108998+0000","last_change":"2024-09-15T01:33:40.109284+0000","last_active":"2024-09-15T01:33:40.108998+0000","last_peered":"2024-09-15T01:33:40.108998+0000","last_clean":"2024-09-15T01:33:40.108998+0000","last_became_active":"2024-09-15T01:32:34.190371+0000","last_became_peered":"2024-09-15T01:32:34.190371+0000","last_unstale":"2024-09-15T01:33:40.108998+0000","last_undegraded":"2024-09-15T01:33:40.108998+0000","last_fullsized":"2024-09-15T01:33:40.108998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:51.142885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384967,"stat_sum":{"num_bytes":13058048,"num_objects":7,"num_object_clones":4,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":26268,"num_write":88,"num_write_kb":13440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"1.0","version":"24'112","reported_seq":218,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T01:33:39.109090+0000","last_change":"2024-09-15T01:32:16.749959+0000","last_active":"2024-09-15T01:33:39.109090+0000","last_peered":"2024-09-15T01:33:39.109090+0000","last_clean":"2024-09-15T01:33:39.109090+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:33:39.109090+0000","last_undegraded":"2024-09-15T01:33:39.109090+0000","last_fullsized":"2024-09-15T01:33:39.109090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:15.451214+0000","last_clean_scrub_stamp":"2024-09-15T01:32:15.451214+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":84,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T01:33:39.106676+0000","last_change":"2024-09-15T01:32:25.147954+0000","last_active":"2024-09-15T01:33:39.106676+0000","last_peered":"2024-09-15T01:33:39.106676+0000","last_clean":"2024-09-15T01:33:39.106676+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:33:39.106676+0000","last_undegraded":"2024-09-15T01:33:39.106676+0000","last_fullsized":"2024-09-15T01:33:39.106676+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.228776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"98'69","reported_seq":317,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T01:33:40.120779+0000","last_change":"2024-09-15T01:33:40.121832+0000","last_active":"2024-09-15T01:33:40.120779+0000","last_peered":"2024-09-15T01:33:40.120779+0000","last_clean":"2024-09-15T01:33:40.120779+0000","last_became_active":"2024-09-15T01:32:57.985433+0000","last_became_peered":"2024-09-15T01:32:57.985433+0000","last_unstale":"2024-09-15T01:33:40.120779+0000","last_undegraded":"2024-09-15T01:33:40.120779+0000","last_fullsized":"2024-09-15T01:33:40.120779+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:55.388361+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0060481290000000002,"stat_sum":{"num_bytes":1622016,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":13454,"num_write":33,"num_write_kb":11648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":5652480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T01:33:39.109190+0000","last_change":"2024-09-15T01:33:23.083602+0000","last_active":"2024-09-15T01:33:39.109190+0000","last_peered":"2024-09-15T01:33:39.109190+0000","last_clean":"2024-09-15T01:33:39.109190+0000","last_became_active":"2024-09-15T01:33:23.083242+0000","last_became_peered":"2024-09-15T01:33:23.083242+0000","last_unstale":"2024-09-15T01:33:39.109190+0000","last_undegraded":"2024-09-15T01:33:39.109190+0000","last_fullsized":"2024-09-15T01:33:39.109190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.932087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8],"acting":[10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1","version":"95'29","reported_seq":335,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T01:33:40.120279+0000","last_change":"2024-09-15T01:33:40.120623+0000","last_active":"2024-09-15T01:33:40.120279+0000","last_peered":"2024-09-15T01:33:40.120279+0000","last_clean":"2024-09-15T01:33:40.120279+0000","last_became_active":"2024-09-15T01:33:12.080783+0000","last_became_peered":"2024-09-15T01:33:12.080783+0000","last_unstale":"2024-09-15T01:33:40.120279+0000","last_undegraded":"2024-09-15T01:33:40.120279+0000","last_fullsized":"2024-09-15T01:33:40.120279+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084944100000000008,"stat_sum":{"num_bytes":1867776,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":11,"num_read_kb":1059,"num_write":11,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":4177920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,2,13],"acting":[4,11,9,5,10,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":84,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T01:33:39.109187+0000","last_change":"2024-09-15T01:32:25.583876+0000","last_active":"2024-09-15T01:33:39.109187+0000","last_peered":"2024-09-15T01:33:39.109187+0000","last_clean":"2024-09-15T01:33:39.109187+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:33:39.109187+0000","last_undegraded":"2024-09-15T01:33:39.109187+0000","last_fullsized":"2024-09-15T01:33:39.109187+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:49.092006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"98'99","reported_seq":299,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T01:33:40.133327+0000","last_change":"2024-09-15T01:33:40.116579+0000","last_active":"2024-09-15T01:33:40.133327+0000","last_peered":"2024-09-15T01:33:40.133327+0000","last_clean":"2024-09-15T01:33:40.133327+0000","last_became_active":"2024-09-15T01:32:34.190384+0000","last_became_peered":"2024-09-15T01:32:34.190384+0000","last_unstale":"2024-09-15T01:33:40.133327+0000","last_undegraded":"2024-09-15T01:33:40.133327+0000","last_fullsized":"2024-09-15T01:33:40.133327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:46.588055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057518439999999999,"stat_sum":{"num_bytes":3375104,"num_objects":9,"num_object_clones":6,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":28,"num_read_kb":3148,"num_write":50,"num_write_kb":5328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":87,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T01:33:39.108432+0000","last_change":"2024-09-15T01:32:25.144113+0000","last_active":"2024-09-15T01:33:39.108432+0000","last_peered":"2024-09-15T01:33:39.108432+0000","last_clean":"2024-09-15T01:33:39.108432+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:33:39.108432+0000","last_undegraded":"2024-09-15T01:33:39.108432+0000","last_fullsized":"2024-09-15T01:33:39.108432+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"96'92","reported_seq":299,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T01:33:40.116948+0000","last_change":"2024-09-15T01:33:40.117288+0000","last_active":"2024-09-15T01:33:40.116948+0000","last_peered":"2024-09-15T01:33:40.116948+0000","last_clean":"2024-09-15T01:33:40.116948+0000","last_became_active":"2024-09-15T01:32:34.170218+0000","last_became_peered":"2024-09-15T01:32:34.170218+0000","last_unstale":"2024-09-15T01:33:40.116948+0000","last_undegraded":"2024-09-15T01:33:40.116948+0000","last_fullsized":"2024-09-15T01:33:40.116948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:41.093067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503304,"stat_sum":{"num_bytes":11993088,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":12303,"num_write":77,"num_write_kb":15056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.5","version":"95'68","reported_seq":262,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T01:33:39.113496+0000","last_change":"2024-09-15T01:33:39.113819+0000","last_active":"2024-09-15T01:33:39.113496+0000","last_peered":"2024-09-15T01:33:39.113496+0000","last_clean":"2024-09-15T01:33:39.113496+0000","last_became_active":"2024-09-15T01:32:34.189813+0000","last_became_peered":"2024-09-15T01:32:34.189813+0000","last_unstale":"2024-09-15T01:33:39.113496+0000","last_undegraded":"2024-09-15T01:33:39.113496+0000","last_fullsized":"2024-09-15T01:33:39.113496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:47.814751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045316699999999998,"stat_sum":{"num_bytes":8306688,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":20,"num_read_kb":6009,"num_write":27,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":81,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T01:33:39.490555+0000","last_change":"2024-09-15T01:32:25.568260+0000","last_active":"2024-09-15T01:33:39.490555+0000","last_peered":"2024-09-15T01:33:39.490555+0000","last_clean":"2024-09-15T01:33:39.490555+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:33:39.490555+0000","last_undegraded":"2024-09-15T01:33:39.490555+0000","last_fullsized":"2024-09-15T01:33:39.490555+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"98'95","reported_seq":295,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T01:33:40.118153+0000","last_change":"2024-09-15T01:33:40.119179+0000","last_active":"2024-09-15T01:33:40.118153+0000","last_peered":"2024-09-15T01:33:40.118153+0000","last_clean":"2024-09-15T01:33:40.118153+0000","last_became_active":"2024-09-15T01:32:34.189644+0000","last_became_peered":"2024-09-15T01:32:34.189644+0000","last_unstale":"2024-09-15T01:33:40.118153+0000","last_undegraded":"2024-09-15T01:33:40.118153+0000","last_fullsized":"2024-09-15T01:33:40.118153+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:02.903663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072801360000000004,"stat_sum":{"num_bytes":8765440,"num_objects":8,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":27,"num_read_kb":13068,"num_write":92,"num_write_kb":10784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":84,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T01:33:39.109195+0000","last_change":"2024-09-15T01:32:25.407396+0000","last_active":"2024-09-15T01:33:39.109195+0000","last_peered":"2024-09-15T01:33:39.109195+0000","last_clean":"2024-09-15T01:33:39.109195+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:33:39.109195+0000","last_undegraded":"2024-09-15T01:33:39.109195+0000","last_fullsized":"2024-09-15T01:33:39.109195+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:32:22.121120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"98'129","reported_seq":464,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T01:33:40.120787+0000","last_change":"2024-09-15T01:33:40.121542+0000","last_active":"2024-09-15T01:33:40.120787+0000","last_peered":"2024-09-15T01:33:40.120787+0000","last_clean":"2024-09-15T01:33:40.120787+0000","last_became_active":"2024-09-15T01:33:12.081147+0000","last_became_peered":"2024-09-15T01:33:12.081147+0000","last_unstale":"2024-09-15T01:33:40.120787+0000","last_undegraded":"2024-09-15T01:33:40.120787+0000","last_fullsized":"2024-09-15T01:33:40.120787+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:52.191150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009983291,"stat_sum":{"num_bytes":13565952,"num_objects":9,"num_object_clones":5,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":37,"num_read_kb":29629,"num_write":58,"num_write_kb":13408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":17530880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":94,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T01:33:39.638484+0000","last_change":"2024-09-15T01:33:39.638452+0000","last_active":"2024-09-15T01:33:39.638484+0000","last_peered":"2024-09-15T01:33:39.638484+0000","last_clean":"2024-09-15T01:33:39.638484+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:33:39.638484+0000","last_undegraded":"2024-09-15T01:33:39.638484+0000","last_fullsized":"2024-09-15T01:33:39.638484+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:39.638404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:39.638404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:59.511423+0000","scrub_duration":0.001362835,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"95'58","reported_seq":348,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T01:33:40.110387+0000","last_change":"2024-09-15T01:33:40.110722+0000","last_active":"2024-09-15T01:33:40.110387+0000","last_peered":"2024-09-15T01:33:40.110387+0000","last_clean":"2024-09-15T01:33:40.110387+0000","last_became_active":"2024-09-15T01:33:12.081225+0000","last_became_peered":"2024-09-15T01:33:12.081225+0000","last_unstale":"2024-09-15T01:33:40.110387+0000","last_undegraded":"2024-09-15T01:33:40.110387+0000","last_fullsized":"2024-09-15T01:33:40.110387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:33.152124+0000","last_clean_scrub_stamp":"2024-09-15T01:32:33.152124+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:53.748998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049399700000000003,"stat_sum":{"num_bytes":5652480,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":6812,"num_write":62,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":6586368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"30","length":"1"},{"start":"32","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":92,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-15T01:33:38.175733+0000","last_change":"2024-09-15T01:33:38.175676+0000","last_active":"2024-09-15T01:33:38.175733+0000","last_peered":"2024-09-15T01:33:38.175733+0000","last_clean":"2024-09-15T01:33:38.175733+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:33:38.175733+0000","last_undegraded":"2024-09-15T01:33:38.175733+0000","last_fullsized":"2024-09-15T01:33:38.175733+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:38.175625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:38.175625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:44.144155+0000","scrub_duration":0.001339526,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":117063680,"num_objects":90,"num_object_clones":47,"num_object_copies":630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":3,"num_read":401,"num_read_kb":216789,"num_write":830,"num_write_kb":167104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":44957696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1439,"ondisk_log_size":1439,"up":112,"acting":112,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313238,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":828204,"kb_used_data":92851788,"kb_used_omap":3352,"kb_used_meta":0,"kb_avail":92851412,"statfs":{"total":95927926784,"available":95079845888,"internally_reserved":385024,"allocated":95080230912,"data_stored":95080230912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3432828,"internal_metadata":0},"hb_peers":[1,2,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.53300000000000003}]},{"osd":2,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":30.641999999999999,"5min":30.641999999999999,"15min":30.641999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":574.26900000000001,"5min":574.26900000000001,"15min":574.26900000000001},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":342.08300000000003}]},{"osd":3,"last update":"Sun Sep 15 01:33:20 2024","interfaces":[{"interface":"back","average":{"1min":32.810000000000002,"5min":32.810000000000002,"15min":32.810000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":644.97699999999998,"5min":644.97699999999998,"15min":644.97699999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":805.39499999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.0699999999999998,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":214.994}]},{"osd":5,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.63900000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.36799999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57399999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":11,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":7.3849999999999998,"5min":7.3849999999999998,"15min":7.3849999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":134.12200000000001,"5min":134.12200000000001,"15min":134.12200000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":846.97400000000005}]},{"osd":12,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.58499999999999996}]},{"osd":13,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.627}]},{"osd":15,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.55600000000000005}]}]},{"osd":9,"up_from":21,"seq":90194313235,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":810876,"kb_used_data":92869004,"kb_used_omap":1311,"kb_used_meta":0,"kb_avail":92868740,"statfs":{"total":95927926784,"available":95097589760,"internally_reserved":270336,"allocated":95097860096,"data_stored":95097860096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1343053,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.74399999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.5709999999999997,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.72999999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.501}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":10.916,"5min":10.916,"15min":10.916},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":13.147,"5min":13.147,"15min":13.147},"last":0.78200000000000003}]},{"osd":10,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":6.4539999999999997,"5min":6.4539999999999997,"15min":6.4539999999999997},"last":0.62}]},{"osd":11,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.9489999999999998,"5min":6.9489999999999998,"15min":6.9489999999999998},"last":0.63500000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":29.687999999999999,"5min":29.687999999999999,"15min":29.687999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":495.95800000000003,"5min":495.95800000000003,"15min":495.95800000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":54.658000000000001,"5min":54.658000000000001,"15min":54.658000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":914.18499999999995,"5min":914.18499999999995,"15min":914.18499999999995},"last":0.754}]},{"osd":13,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":13.132999999999999,"5min":13.132999999999999,"15min":13.132999999999999},"last":0.44700000000000001}]},{"osd":14,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68300000000000005}]},{"osd":15,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.82399999999999995}]}]},{"osd":3,"up_from":21,"seq":90194313236,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":809060,"kb_used_data":92870844,"kb_used_omap":1742,"kb_used_meta":0,"kb_avail":92870556,"statfs":{"total":95927926784,"available":95099449344,"internally_reserved":294912,"allocated":95099744256,"data_stored":95099744256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1784370,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.76300000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.439}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.60199999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.4720000000000004,"5min":7.4720000000000004,"15min":7.4720000000000004},"last":0.68500000000000005},{"interface":"front","average":{"1min":9.3040000000000003,"5min":9.3040000000000003,"15min":9.3040000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":182.87,"5min":182.87,"15min":182.87},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.67100000000000004}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":24.288,"5min":24.288,"15min":24.288},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":500.678,"5min":500.678,"15min":500.678},"last":0.55500000000000005},{"interface":"front","average":{"1min":35.298999999999999,"5min":35.298999999999999,"15min":35.298999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":500.79899999999998,"5min":500.79899999999998,"15min":500.79899999999998},"last":0.73399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":12,"last update":"Sun Sep 15 01:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.622},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.75}]},{"osd":13,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":7.4500000000000002,"5min":7.4500000000000002,"15min":7.4500000000000002},"last":0.78700000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.96299999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]}]},{"osd":0,"up_from":21,"seq":90194313237,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":844520,"kb_used_data":92835520,"kb_used_omap":4578,"kb_used_meta":0,"kb_avail":92835096,"statfs":{"total":95927926784,"available":95063138304,"internally_reserved":434176,"allocated":95063572480,"data_stored":95063572480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4687896,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.58999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":39.392000000000003,"5min":39.392000000000003,"15min":39.392000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":814.68299999999999,"5min":814.68299999999999,"15min":814.68299999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.434},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.33600000000000002}]},{"osd":6,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.2769999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.264}]},{"osd":7,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.45100000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.56100000000000005}]},{"osd":10,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.377},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":1.3089999999999999}]},{"osd":12,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.41799999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":25.651,"5min":25.651,"15min":25.651},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":501.07400000000001,"5min":501.07400000000001,"15min":501.07400000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":44.5,"5min":44.5,"15min":44.5},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":501.08800000000002,"5min":501.08800000000002,"15min":501.08800000000002},"last":0.60899999999999999}]},{"osd":15,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.403}]}]},{"osd":13,"up_from":21,"seq":90194313236,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":822208,"kb_used_data":92857772,"kb_used_omap":3664,"kb_used_meta":0,"kb_avail":92857408,"statfs":{"total":95927926784,"available":95085985792,"internally_reserved":372736,"allocated":95086358528,"data_stored":95086358528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3752262,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.36299999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.1129999999999995,"5min":8.1129999999999995,"15min":8.1129999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":8.1430000000000007,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":0.70799999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":19.710000000000001,"5min":19.710000000000001,"15min":19.710000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":342.56099999999998,"5min":342.56099999999998,"15min":342.56099999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":28.227,"5min":28.227,"15min":28.227},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":495.40800000000002,"5min":495.40800000000002,"15min":495.40800000000002},"last":0.69499999999999995}]},{"osd":4,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.7960000000000003,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.7619999999999996,"5min":5.7619999999999996,"15min":5.7619999999999996},"last":0.65400000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40699999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":18.945,"5min":18.945,"15min":18.945},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":293.34800000000001,"5min":293.34800000000001,"15min":293.34800000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.748}]},{"osd":7,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.4779999999999998,"5min":6.4779999999999998,"15min":6.4779999999999998},"last":0.627},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":5.726,"5min":5.726,"15min":5.726},"last":0.67000000000000004}]},{"osd":8,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":44.985999999999997,"5min":44.985999999999997,"15min":44.985999999999997},"last":0.48099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71599999999999997}]},{"osd":14,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.78000000000000003}]},{"osd":15,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.68400000000000005}]}]},{"osd":2,"up_from":21,"seq":90194313235,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":830932,"kb_used_data":92849060,"kb_used_omap":3550,"kb_used_meta":0,"kb_avail":92848684,"statfs":{"total":95927926784,"available":95077052416,"internally_reserved":385024,"allocated":95077437440,"data_stored":95077437440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3635760,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.38}]},{"osd":3,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.33400000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":6.6680000000000001,"5min":6.6680000000000001,"15min":6.6680000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":110.032,"5min":110.032,"15min":110.032},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.57299999999999995}]},{"osd":8,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.58999999999999997}]},{"osd":9,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.36299999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67900000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.63100000000000001}]},{"osd":13,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.51900000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.47699999999999998}]}]},{"osd":15,"up_from":21,"seq":90194313236,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":822536,"kb_used_data":92857356,"kb_used_omap":2781,"kb_used_meta":0,"kb_avail":92857080,"statfs":{"total":95927926784,"available":95085649920,"internally_reserved":282624,"allocated":95085932544,"data_stored":95085932544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2848135,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.372},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.52000000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.34799999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.441},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71399999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":5.8620000000000001,"5min":5.8620000000000001,"15min":5.8620000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":90.775999999999996,"5min":90.775999999999996,"15min":90.775999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.73999999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:33:36 2024","interfaces":[{"interface":"back","average":{"1min":50.158999999999999,"5min":50.158999999999999,"15min":50.158999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":894.27700000000004,"5min":894.27700000000004,"15min":894.27700000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":49.308,"5min":49.308,"15min":49.308},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":878.12199999999996,"5min":878.12199999999996,"15min":878.12199999999996},"last":0.48899999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.50900000000000001},{"interface":"front","average":{"1min":34.816000000000003,"5min":34.816000000000003,"15min":34.816000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":542.06600000000003,"5min":542.06600000000003,"15min":542.06600000000003},"last":0.55200000000000005}]},{"osd":13,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.66100000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.72799999999999998},{"interface":"front","average":{"1min":15.124000000000001,"5min":15.124000000000001,"15min":15.124000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":246.56100000000001,"5min":246.56100000000001,"15min":246.56100000000001},"last":0.77400000000000002}]}]},{"osd":4,"up_from":21,"seq":90194313236,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":825844,"kb_used_data":92854140,"kb_used_omap":2521,"kb_used_meta":0,"kb_avail":92853772,"statfs":{"total":95927926784,"available":95082262528,"internally_reserved":376832,"allocated":95082639360,"data_stored":95082639360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2582366,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":570,"commit_latency_ns":0,"apply_latency_ns":570000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":17.896999999999998,"5min":17.896999999999998,"15min":17.896999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":314.661,"5min":314.661,"15min":314.661},"last":0.60399999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55700000000000005}]},{"osd":3,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.76700000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.40300000000000002},{"interface":"front","average":{"1min":19.617999999999999,"5min":19.617999999999999,"15min":19.617999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":361.452,"5min":361.452,"15min":361.452},"last":0.57699999999999996}]},{"osd":6,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.68200000000000005}]},{"osd":7,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.71799999999999997}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.621},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.78300000000000003}]},{"osd":9,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.73999999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.872}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":12,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.752}]},{"osd":13,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.85199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.89300000000000002}]}]},{"osd":5,"up_from":21,"seq":90194313235,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":831008,"kb_used_data":92849108,"kb_used_omap":2540,"kb_used_meta":0,"kb_avail":92848608,"statfs":{"total":95927926784,"available":95076974592,"internally_reserved":512000,"allocated":95077486592,"data_stored":95077486592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2600995,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.34300000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.307},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.23999999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.39900000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.41899999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.36299999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.526},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.47499999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.45100000000000001}]},{"osd":13,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.439}]},{"osd":14,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.1899999999999999},{"interface":"front","average":{"1min":24.149000000000001,"5min":24.149000000000001,"15min":24.149000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":492.697,"5min":492.697,"15min":492.697},"last":3.9990000000000001}]},{"osd":15,"last update":"Sun Sep 15 01:33:26 2024","interfaces":[{"interface":"back","average":{"1min":13.939,"5min":13.939,"15min":13.939},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":291.12599999999998,"5min":291.12599999999998,"15min":291.12599999999998},"last":1.6399999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.63100000000000001}]}]},{"osd":6,"up_from":21,"seq":90194313235,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":834600,"kb_used_data":92845392,"kb_used_omap":2709,"kb_used_meta":0,"kb_avail":92845016,"statfs":{"total":95927926784,"available":95073296384,"internally_reserved":385024,"allocated":95073681408,"data_stored":95073681408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2774210,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":4.3449999999999998,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.63}]},{"osd":3,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.68600000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39100000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.47899999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.752},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.72799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.76100000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.54400000000000004}]},{"osd":11,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.56200000000000006}]},{"osd":12,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.379},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":5.8470000000000004,"5min":5.8470000000000004,"15min":5.8470000000000004},"last":0.64300000000000002}]},{"osd":13,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.66800000000000004}]}]},{"osd":7,"up_from":21,"seq":90194313236,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":813056,"kb_used_data":92866820,"kb_used_omap":2177,"kb_used_meta":0,"kb_avail":92866560,"statfs":{"total":95927926784,"available":95095357440,"internally_reserved":266240,"allocated":95095623680,"data_stored":95095623680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2230247,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":59.375,"5min":59.375,"15min":59.375},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":874.39499999999998,"5min":874.39499999999998,"15min":874.39499999999998},"last":0.65800000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":4.7350000000000003,"5min":4.7350000000000003,"15min":4.7350000000000003},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.32700000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":5.6539999999999999,"5min":5.6539999999999999,"15min":5.6539999999999999},"last":0.63500000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68899999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":5.3860000000000001,"5min":5.3860000000000001,"15min":5.3860000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":62.991999999999997,"5min":62.991999999999997,"15min":62.991999999999997},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.70299999999999996},{"interface":"front","average":{"1min":54.116,"5min":54.116,"15min":54.116},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":792.73599999999999,"5min":792.73599999999999,"15min":792.73599999999999},"last":0.68000000000000005}]},{"osd":13,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":8.1489999999999991,"5min":8.1489999999999991,"15min":8.1489999999999991},"last":0.52400000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":52.212000000000003,"5min":52.212000000000003,"15min":52.212000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":773.18799999999999,"5min":773.18799999999999,"15min":773.18799999999999},"last":0.73399999999999999}]},{"osd":15,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.439},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53800000000000003}]}]},{"osd":8,"up_from":21,"seq":90194313235,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":828476,"kb_used_data":92851508,"kb_used_omap":3226,"kb_used_meta":0,"kb_avail":92851140,"statfs":{"total":95927926784,"available":95079567360,"internally_reserved":376832,"allocated":95079944192,"data_stored":95079944192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3303792,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.36099999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":24.794,"5min":24.794,"15min":24.794},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":413.66800000000001,"5min":413.66800000000001,"15min":413.66800000000001},"last":0.497}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":800.48500000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34699999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.379}]},{"osd":6,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.437},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.45200000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.41899999999999998}]},{"osd":9,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.42899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.56299999999999994}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.51900000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.58299999999999996}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313235,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":815024,"kb_used_data":92864984,"kb_used_omap":2760,"kb_used_meta":0,"kb_avail":92864592,"statfs":{"total":95927926784,"available":95093342208,"internally_reserved":401408,"allocated":95093743616,"data_stored":95093743616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2826693,"internal_metadata":0},"hb_peers":[3,5,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.899},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":5,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.9500000000000002,"5min":3.9500000000000002,"15min":3.9500000000000002},"last":3.9750000000000001},{"interface":"front","average":{"1min":4.4180000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":64.938999999999993,"5min":64.938999999999993,"15min":64.938999999999993},"last":3.96}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.8959999999999999,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.65800000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":9,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":55.476999999999997,"5min":55.476999999999997,"15min":55.476999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":500.92099999999999,"5min":500.92099999999999,"15min":500.92099999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.76900000000000002}]},{"osd":11,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.628}]},{"osd":12,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":4.2949999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.64300000000000002}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.4460000000000002},{"interface":"front","average":{"1min":36.435000000000002,"5min":36.435000000000002,"15min":36.435000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":641.649,"5min":641.649,"15min":641.649},"last":2.4740000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.752},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.83699999999999997}]},{"osd":15,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.0670000000000002,"5min":6.0670000000000002,"15min":6.0670000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.78400000000000003}]}]},{"osd":11,"up_from":21,"seq":90194313235,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":822688,"kb_used_data":92857244,"kb_used_omap":2616,"kb_used_meta":0,"kb_avail":92856928,"statfs":{"total":95927926784,"available":95085494272,"internally_reserved":323584,"allocated":95085817856,"data_stored":95085817856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2679378,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":46.469000000000001,"5min":46.469000000000001,"15min":46.469000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":873.95500000000004,"5min":873.95500000000004,"15min":873.95500000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.954,"5min":1.954,"15min":1.954},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":28.431999999999999,"5min":28.431999999999999,"15min":28.431999999999999},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.50600000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59399999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.63100000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.66300000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":10,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":0.71499999999999997}]},{"osd":12,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":5.5069999999999997,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":5.3120000000000003,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":0.80500000000000005}]},{"osd":13,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.752},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.72899999999999998}]},{"osd":14,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":16.126000000000001,"5min":16.126000000000001,"15min":16.126000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":310.78199999999998,"5min":310.78199999999998,"15min":310.78199999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":0.79200000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":12,"up_from":21,"seq":90194313236,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":807540,"kb_used_data":92872228,"kb_used_omap":1291,"kb_used_meta":0,"kb_avail":92872076,"statfs":{"total":95927926784,"available":95101005824,"internally_reserved":155648,"allocated":95101161472,"data_stored":95101161472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1322137,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.35499999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":8.0050000000000008,"5min":8.0050000000000008,"15min":8.0050000000000008},"last":0.497},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":0.64300000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.56999999999999995}]},{"osd":3,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.435},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.50600000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":24.937000000000001,"5min":24.937000000000001,"15min":24.937000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":488.69,"5min":488.69,"15min":488.69},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5419999999999998,"5min":3.5419999999999998,"15min":3.5419999999999998},"last":0.40000000000000002}]},{"osd":6,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.6309999999999998,"5min":3.6309999999999998,"15min":3.6309999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.67200000000000004}]},{"osd":11,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.46899999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.63},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.68600000000000005}]},{"osd":14,"last update":"Sun Sep 15 01:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.61399999999999999}]},{"osd":15,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.52900000000000003}]}]},{"osd":1,"up_from":21,"seq":90194313236,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":825108,"kb_used_data":92855008,"kb_used_omap":2997,"kb_used_meta":0,"kb_avail":92854508,"statfs":{"total":95927926784,"available":95083016192,"internally_reserved":512000,"allocated":95083528192,"data_stored":95083528192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3068929,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":2,"commit_latency_ns":0,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.55300000000000005}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.498},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.52100000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.68799999999999994}]},{"osd":4,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":35.843000000000004,"5min":35.843000000000004,"15min":35.843000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":635.46400000000006,"5min":635.46400000000006,"15min":635.46400000000006},"last":0.47499999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.441},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":6.4690000000000003,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.45700000000000002}]},{"osd":6,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.53500000000000003}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.64600000000000002}]},{"osd":8,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.50600000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":27.920000000000002,"5min":27.920000000000002,"15min":27.920000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":9.9450000000000003,"5min":9.9450000000000003,"15min":9.9450000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":166.16200000000001,"5min":166.16200000000001,"15min":166.16200000000001},"last":0.67200000000000004}]},{"osd":12,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.65700000000000003}]}]}],"pool_statfs":[]}} 2024-09-15T01:33:44.741 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-15T01:33:44.741 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-15T01:33:44.742 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:44.742 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:44.751 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:33:44.776 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:33:44.790 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:44.788+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:44.847 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:44.847 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:44.856 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:33:44.891 INFO:tasks.ceph.osd.12.smithi042.stderr:2024-09-15T01:33:44.890+0000 7f65ac1f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 28842) UID: 0 2024-09-15T01:33:44.911 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-15T01:33:44.912 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:33:44.944 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-15T01:33:44.944 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:33:44.952 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:44.953 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:44.961 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:33:44.977 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-15T01:33:44.977 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:33:44.992 INFO:tasks.ceph.osd.14.smithi152.stderr:2024-09-15T01:33:44.991+0000 7f06e51f9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 28728) UID: 0 2024-09-15T01:33:45.009 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-15T01:33:45.010 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:33:45.032 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-15T01:33:45.033 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:33:45.051 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-15T01:33:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:33:45.058 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:45.058 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:45.067 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:33:45.093 INFO:tasks.ceph.osd.15.smithi175.stderr:2024-09-15T01:33:45.091+0000 7fa501bf9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 28833) UID: 0 2024-09-15T01:33:45.131 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'unique_pool_0' removed 2024-09-15T01:33:45.144 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-15T01:33:45.154 INFO:tasks.thrashosds:joining thrashosds 2024-09-15T01:33:45.154 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:33:45.165 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-15T01:33:45.165 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:33:45.167 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-15T01:33:45.168 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:33:45.174 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:33:45.187 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-15T01:33:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:33:45.205 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8: {} 2024-09-15T01:33:45.206 INFO:teuthology.orchestra.run.smithi042.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:33:45.224 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9: {} 2024-09-15T01:33:45.225 INFO:teuthology.orchestra.run.smithi042.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:33:45.274 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:33:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:33:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:33:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 70/69 ECSubWrite(tid=71, reqid=client.4477.0:636, at_version=70'43, trim_to=0'0, roll_forward_to=70'40))", 2024-09-15T01:33:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:12.102080+0000", 2024-09-15T01:33:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.169297092999997, 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050273510000000002, 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.102080+0000", 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.102080+0000", 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.102081+0000", 2024-09-15T01:33:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6899999999999996e-07 2024-09-15T01:33:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103542+0000", 2024-09-15T01:33:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014608170000000001 2024-09-15T01:33:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103579+0000", 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7376999999999998e-05 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103587+0000", 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2640000000000008e-06 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103636+0000", 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8334000000000002e-05 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103646+0000", 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0010000000000001e-05 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103944+0000", 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029775800000000002 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.103952+0000", 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7900000000000005e-06 2024-09-15T01:33:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104702+0000", 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00074918900000000004 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.104723+0000", 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1610000000000001e-05 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.107107+0000", 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023842329999999999 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 71/69 ECSubWrite(tid=51, reqid=client.4477.0:678, at_version=71'33, trim_to=0'0, roll_forward_to=70'31))", 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.271454+0000", 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.999922726000001, 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050481479999999997, 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271454+0000", 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271454+0000", 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271455+0000", 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6599999999999996e-07 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271510+0000", 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4975999999999997e-05 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271546+0000", 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6066999999999999e-05 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271553+0000", 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2439999999999996e-06 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271623+0000", 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9549999999999996e-05 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271634+0000", 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1399e-05 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271824+0000", 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000189877 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271834+0000", 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5990000000000008e-06 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271871+0000", 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7718e-05 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.271888+0000", 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6881e-05 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.276503+0000", 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046142709999999997 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 71/69 ECSubWrite(tid=175, reqid=client.4477.0:697, at_version=71'58, trim_to=0'0, roll_forward_to=71'55))", 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.307088+0000", 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.964288673999999, 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049056919999999997, 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307088+0000", 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307088+0000", 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307090+0000", 2024-09-15T01:33:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.099e-06 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307121+0000", 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1538000000000002e-05 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307175+0000", 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3906999999999997e-05 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307183+0000", 2024-09-15T01:33:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7549999999999994e-06 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307247+0000", 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4070000000000002e-05 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307257+0000", 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9729999999999992e-06 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307362+0000", 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010547099999999999 2024-09-15T01:33:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307371+0000", 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.439e-06 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307402+0000", 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1121999999999998e-05 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.307425+0000", 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3598999999999999e-05 2024-09-15T01:33:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.311994+0000", 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045687189999999997 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s6 71/69 ECSubWrite(tid=73, reqid=client.4477.0:701, at_version=71'45, trim_to=0'0, roll_forward_to=70'43))", 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.321470+0000", 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.949906999, 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012701758, 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.321470+0000", 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.321470+0000", 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.321470+0000", 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3299999999999998e-07 2024-09-15T01:33:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.323991+0000", 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025208689999999998 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324019+0000", 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7208000000000001e-05 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324025+0000", 2024-09-15T01:33:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1959999999999996e-06 2024-09-15T01:33:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324092+0000", 2024-09-15T01:33:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7300999999999998e-05 2024-09-15T01:33:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324105+0000", 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2617e-05 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324615+0000", 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051016099999999997 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324624+0000", 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6039999999999999e-06 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324947+0000", 2024-09-15T01:33:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032275400000000002 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.324963+0000", 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5491000000000001e-05 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.334172+0000", 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0092092240000000002 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 76/69 ECSubWrite(tid=204, reqid=client.4477.0:748, at_version=76'69, trim_to=0'0, roll_forward_to=76'67))", 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:18.103204+0000", 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.168172798000001, 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065162520000000002, 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.103204+0000", 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.103204+0000", 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.103205+0000", 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.61e-07 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105300+0000", 2024-09-15T01:33:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020949359999999999 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105325+0000", 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5287999999999999e-05 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105330+0000", 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6959999999999996e-06 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105379+0000", 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8591000000000003e-05 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105389+0000", 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0919e-05 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105775+0000", 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038574300000000002 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105785+0000", 2024-09-15T01:33:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0015e-05 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.105984+0000", 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019886700000000001 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.106001+0000", 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6413999999999999e-05 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:18.109721+0000", 2024-09-15T01:33:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037200219999999999 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 80/69 ECSubWrite(tid=209, reqid=client.4477.0:795, at_version=80'74, trim_to=0'0, roll_forward_to=78'71))", 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:22.083898+0000", 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.187479173, 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005861008, 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.083898+0000", 2024-09-15T01:33:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.083898+0000", 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.083899+0000", 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9800000000000003e-07 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.083924+0000", 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5009999999999999e-05 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.083986+0000", 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2021999999999997e-05 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.083993+0000", 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8179999999999998e-06 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084033+0000", 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0630999999999998e-05 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084042+0000", 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9269999999999994e-06 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084129+0000", 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7101000000000005e-05 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084142+0000", 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2359e-05 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084190+0000", 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8528e-05 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084304+0000", 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114367 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.089759+0000", 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054544470000000003 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 80/69 ECSubWrite(tid=210, reqid=client.4477.0:796, at_version=80'75, trim_to=0'0, roll_forward_to=78'71))", 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:22.083932+0000", 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.187444907, 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060446629999999996, 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.083932+0000", 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.083932+0000", 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.083933+0000", 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0499999999999999e-07 2024-09-15T01:33:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.083947+0000", 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4433999999999999e-05 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084008+0000", 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0514999999999999e-05 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084012+0000", 2024-09-15T01:33:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8790000000000002e-06 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084164+0000", 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015121700000000001 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084171+0000", 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084277+0000", 2024-09-15T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106776 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084286+0000", 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7129999999999994e-06 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084321+0000", 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5271000000000002e-05 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.084341+0000", 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9199e-05 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.089977+0000", 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056362080000000002 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 80/69 ECSubWrite(tid=211, reqid=client.4477.0:796, at_version=0'0, trim_to=0'0, roll_forward_to=80'75))", 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:22.085203+0000", 2024-09-15T01:33:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.186174449999999, 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050108280000000002, 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.085203+0000", 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.085203+0000", 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.085203+0000", 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999995e-07 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.085220+0000", 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6948000000000001e-05 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.085257+0000", 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6925000000000003e-05 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.085264+0000", 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7070000000000003e-06 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.085326+0000", 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2147e-05 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.085336+0000", 2024-09-15T01:33:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7850000000000003e-06 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.085468+0000", 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000132082 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.085486+0000", 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8219e-05 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.085529+0000", 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2778999999999998e-05 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.085550+0000", 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0661000000000001e-05 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:22.090213+0000", 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046638640000000002 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.9s6 85/76 ECSubWrite(tid=337, reqid=client.4477.0:808, at_version=85'66, trim_to=0'0, roll_forward_to=83'65))", 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:27.102580+0000", 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.168796675999999, 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082243469999999999, 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.102580+0000", 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.102580+0000", 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.102581+0000", 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-07 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.102598+0000", 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7224999999999999e-05 2024-09-15T01:33:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.102651+0000", 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3405000000000001e-05 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.102668+0000", 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6351e-05 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.102732+0000", 2024-09-15T01:33:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4900000000000005e-05 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.102745+0000", 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2265e-05 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.102887+0000", 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014270799999999999 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.102900+0000", 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2502e-05 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.103080+0000", 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018000700000000001 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.103106+0000", 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6230000000000001e-05 2024-09-15T01:33:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110805+0000", 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076986240000000003 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.9s6 85/76 ECSubWrite(tid=342, reqid=osd.11.0:342, at_version=85'69, trim_to=0'0, roll_forward_to=83'65))", 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:27.106304+0000", 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.165073137, 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049920320000000004, 2024-09-15T01:33:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.106304+0000", 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.106304+0000", 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.106304+0000", 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.65e-07 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.106327+0000", 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3252e-05 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.108582+0000", 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0022544610000000001 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.108588+0000", 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5490000000000003e-06 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.108647+0000", 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8130000000000001e-05 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.108657+0000", 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0512e-05 2024-09-15T01:33:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.109674+0000", 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010166419999999999 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.109755+0000", 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1196e-05 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110315+0000", 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055974900000000001 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110329+0000", 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4722000000000001e-05 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.111296+0000", 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000966554 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 85/69 ECSubWrite(tid=222, reqid=osd.4.0:222, at_version=85'85, trim_to=0'0, roll_forward_to=83'83))", 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:27.107984+0000", 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.16339352, 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061328729999999996, 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.107984+0000", 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.107984+0000", 2024-09-15T01:33:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.107984+0000", 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2499999999999999e-07 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.108036+0000", 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1730000000000001e-05 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110052+0000", 2024-09-15T01:33:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020163720000000002 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110067+0000", 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4732000000000001e-05 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110127+0000", 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0197e-05 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110143+0000", 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6149000000000001e-05 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110719+0000", 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057611999999999995 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110728+0000", 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1020000000000003e-06 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.112193+0000", 2024-09-15T01:33:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014647499999999999 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.112217+0000", 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3804000000000001e-05 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.114116+0000", 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018996919999999999 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 85/69 ECSubWrite(tid=223, reqid=osd.4.0:223, at_version=85'87, trim_to=0'0, roll_forward_to=83'83))", 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:27.108039+0000", 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.163337928000001, 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074323089999999998, 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.108039+0000", 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.108039+0000", 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.108040+0000", 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.108081+0000", 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1712999999999999e-05 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110214+0000", 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021327920000000001 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110220+0000", 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3239999999999999e-06 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110793+0000", 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057220600000000002 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.110803+0000", 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0893000000000001e-05 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.111115+0000", 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031201800000000001 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.112197+0000", 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010818830000000001 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.112240+0000", 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2373000000000002e-05 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.112251+0000", 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1188e-05 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.115471+0000", 2024-09-15T01:33:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032205630000000001 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s2 86/69 ECSubWrite(tid=226, reqid=client.4477.0:830, at_version=86'90, trim_to=0'0, roll_forward_to=85'87))", 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:28.113525+0000", 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.157852381000001, 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051282089999999999, 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.113525+0000", 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.113525+0000", 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.113526+0000", 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1399999999999995e-07 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.113543+0000", 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7224e-05 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.113571+0000", 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8501999999999999e-05 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.113577+0000", 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6710000000000004e-06 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.113619+0000", 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2354999999999997e-05 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.113627+0000", 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1149999999999994e-06 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.113771+0000", 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014378500000000001 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.113779+0000", 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4220000000000003e-06 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.113823+0000", 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4697999999999998e-05 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.113837+0000", 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799e-05 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.118653+0000", 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048157240000000004 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 86/69 ECSubWrite(tid=63, reqid=client.4477.0:838, at_version=86'43, trim_to=0'0, roll_forward_to=83'41))", 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:28.126883+0000", 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.144493674, 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0092316989999999995, 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.126883+0000", 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.126883+0000", 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.126884+0000", 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.22e-07 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.126907+0000", 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3180000000000002e-05 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.126993+0000", 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5860999999999997e-05 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.126999+0000", 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4239999999999998e-06 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127054+0000", 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4982000000000003e-05 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127064+0000", 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5599999999999999e-06 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127332+0000", 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026824299999999999 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127377+0000", 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4844e-05 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127429+0000", 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2342000000000001e-05 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127447+0000", 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7904999999999999e-05 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.136115+0000", 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0086681359999999999 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 86/69 ECSubWrite(tid=64, reqid=client.4477.0:839, at_version=86'44, trim_to=0'0, roll_forward_to=83'41))", 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:28.127491+0000", 2024-09-15T01:33:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.143885896, 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017026632999999999, 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127491+0000", 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127491+0000", 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127492+0000", 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1399999999999996e-07 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127515+0000", 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2745999999999999e-05 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127559+0000", 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4344000000000001e-05 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127566+0000", 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9480000000000002e-06 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127615+0000", 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9428000000000002e-05 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127624+0000", 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5690000000000005e-06 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127728+0000", 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103769 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127747+0000", 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8949000000000001e-05 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127792+0000", 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4951999999999999e-05 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.127822+0000", 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9950999999999998e-05 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.144518+0000", 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016696163 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 86/69 ECSubWrite(tid=65, reqid=client.4477.0:840, at_version=86'45, trim_to=0'0, roll_forward_to=83'41))", 2024-09-15T01:33:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:28.128714+0000", 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.142663485, 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015968613999999999, 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.128714+0000", 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.128714+0000", 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.128714+0000", 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4099999999999998e-07 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.128729+0000", 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4888e-05 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.128754+0000", 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4939e-05 2024-09-15T01:33:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.128761+0000", 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5479999999999998e-06 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.128801+0000", 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0592e-05 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.128809+0000", 2024-09-15T01:33:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8029999999999997e-06 2024-09-15T01:33:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.128884+0000", 2024-09-15T01:33:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4386000000000002e-05 2024-09-15T01:33:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.128892+0000", 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2519999999999999e-06 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.128923+0000", 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1647999999999999e-05 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.128936+0000", 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2908000000000001e-05 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.144682+0000", 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015745908999999999 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 86/69 ECSubWrite(tid=229, reqid=client.4477.0:845, at_version=86'37, trim_to=0'0, roll_forward_to=83'35))", 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:28.130440+0000", 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.140937433000001, 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029285528000000002, 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.130440+0000", 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.130440+0000", 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.130440+0000", 2024-09-15T01:33:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:33:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.130462+0000", 2024-09-15T01:33:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1382999999999998e-05 2024-09-15T01:33:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.130497+0000", 2024-09-15T01:33:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5209000000000001e-05 2024-09-15T01:33:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.130503+0000", 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6980000000000003e-06 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.130549+0000", 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6659999999999997e-05 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.130558+0000", 2024-09-15T01:33:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1449999999999997e-06 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.131912+0000", 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013535019999999999 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.131922+0000", 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9569999999999997e-06 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132012+0000", 2024-09-15T01:33:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0556000000000005e-05 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132031+0000", 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8101000000000002e-05 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.159725+0000", 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027694659 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 86/69 ECSubWrite(tid=66, reqid=client.4477.0:840, at_version=0'0, trim_to=0'0, roll_forward_to=86'45))", 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:28.131893+0000", 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.139483907999999, 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012979187999999999, 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.131893+0000", 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.131893+0000", 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.131894+0000", 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8100000000000001e-07 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.131906+0000", 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1634000000000001e-05 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.131942+0000", 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6442000000000001e-05 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.131951+0000", 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6510000000000007e-06 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132005+0000", 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3876999999999999e-05 2024-09-15T01:33:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132015+0000", 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0378e-05 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132125+0000", 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109758 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132134+0000", 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9910000000000008e-06 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132174+0000", 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9931000000000001e-05 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132188+0000", 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4311000000000001e-05 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.144872+0000", 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012684334 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 86/69 ECSubWrite(tid=230, reqid=client.4477.0:845, at_version=0'0, trim_to=0'0, roll_forward_to=86'37))", 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:28.132452+0000", 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.138924939999999, 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027501682999999999, 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132452+0000", 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132452+0000", 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132453+0000", 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999995e-07 2024-09-15T01:33:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132462+0000", 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7409999999999998e-06 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132497+0000", 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.553e-05 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132506+0000", 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4100000000000008e-06 2024-09-15T01:33:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132564+0000", 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8403999999999999e-05 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132575+0000", 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1249e-05 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132665+0000", 2024-09-15T01:33:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9888000000000004e-05 2024-09-15T01:33:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132673+0000", 2024-09-15T01:33:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6510000000000001e-06 2024-09-15T01:33:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132702+0000", 2024-09-15T01:33:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9524000000000001e-05 2024-09-15T01:33:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132712+0000", 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5650000000000007e-06 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.159954+0000", 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027242010000000001 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.9s6 88/76 ECSubWrite(tid=373, reqid=client.4477.0:871, at_version=88'71, trim_to=0'0, roll_forward_to=85'69))", 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:30.158433+0000", 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.112944001000001, 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048893280000000001, 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.158433+0000", 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.158433+0000", 2024-09-15T01:33:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.158433+0000", 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.158458+0000", 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4899999999999999e-05 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.158503+0000", 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4535e-05 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.158510+0000", 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6310000000000003e-06 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.158571+0000", 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0849e-05 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.158582+0000", 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1122e-05 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.158935+0000", 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000353092 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.159140+0000", 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020423699999999999 2024-09-15T01:33:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.159182+0000", 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2352999999999999e-05 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.159472+0000", 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029042100000000002 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:30.163322+0000", 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038499770000000001 2024-09-15T01:33:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.383 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:45.384 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10: {} 2024-09-15T01:33:45.384 INFO:teuthology.orchestra.run.smithi042.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:33:45.385 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-15T01:33:45.407 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11: {} 2024-09-15T01:33:45.408 INFO:teuthology.orchestra.run.smithi042.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:33:45.441 INFO:teuthology.orchestra.run.smithi042.stdout:osd.12: {} 2024-09-15T01:33:45.441 INFO:teuthology.orchestra.run.smithi042.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-15T01:33:45.460 INFO:teuthology.orchestra.run.smithi042.stdout:osd.13: {} 2024-09-15T01:33:45.460 INFO:teuthology.orchestra.run.smithi042.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-15T01:33:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-15T01:33:45.464+0000 7f2f95d9b640 0 -- 172.21.15.42:0/3482661047 >> [v2:172.21.15.152:6823/28734,v1:172.21.15.152:6825/28734] conn(0x7f2f981948c0 msgr2=0x7f2f98196ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-15T01:33:45.485 INFO:teuthology.orchestra.run.smithi042.stdout:osd.14: {} 2024-09-15T01:33:45.486 INFO:teuthology.orchestra.run.smithi042.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-15T01:33:45.487 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-15T01:33:45.487 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:45.496 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-15T01:33:45.508 INFO:teuthology.orchestra.run.smithi042.stdout:osd.15: {} 2024-09-15T01:33:45.508 INFO:teuthology.orchestra.run.smithi042.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-15T01:33:45.591 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-15T01:33:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-15T01:33:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-15T01:33:45.592 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:45.600 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-15T01:33:45.699 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-15T01:33:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-15T01:33:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-15T01:33:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.217437141000005, 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27210968699999999, 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4181", 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/3658765257", 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479164+0000", 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479166+0000", 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4729999999999999e-06 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479172+0000", 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7460000000000003e-06 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479186+0000", 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4011e-05 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479211+0000", 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4658e-05 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479249+0000", 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7706000000000001e-05 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.479255+0000", 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9669999999999998e-06 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750042+0000", 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27078760899999998 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750204+0000", 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016171800000000001 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750337+0000", 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000132674 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750470+0000", 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013314699999999999 2024-09-15T01:33:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750485+0000", 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5288e-05 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750515+0000", 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0179e-05 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.750582+0000", 2024-09-15T01:33:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6890999999999994e-05 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751212+0000", 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062971900000000005 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751249+0000", 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7072e-05 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:16.751274+0000", 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4828999999999999e-05 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 34/31 ECSubWrite(tid=13, reqid=client.4477.0:128, at_version=34'8, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.419287350000005, 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.101528777, 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277314+0000", 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7999999999999995e-07 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.277326+0000", 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.184e-05 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378494+0000", 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.101167374 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378498+0000", 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1150000000000004e-06 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378565+0000", 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7354999999999995e-05 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378574+0000", 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000007e-06 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378662+0000", 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8287000000000004e-05 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378672+0000", 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3840000000000003e-06 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378705+0000", 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3506000000000001e-05 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378722+0000", 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6449000000000001e-05 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:36.378843+0000", 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000120937 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.251636+0000", 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.444964896999998, 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024080965999999999, 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251636+0000", 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251636+0000", 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251637+0000", 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7800000000000001e-07 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251655+0000", 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8062000000000001e-05 2024-09-15T01:33:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251686+0000", 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0723999999999999e-05 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251690+0000", 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0049999999999998e-06 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251718+0000", 2024-09-15T01:33:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8486000000000001e-05 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.251725+0000", 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.252412+0000", 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068728599999999995 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.252417+0000", 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.253911+0000", 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001493858 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.253956+0000", 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4316000000000001e-05 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275717+0000", 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021761656000000001 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:282 3.3s0 3:c4c60c95:::smithi17529310-28:head [delete] snapc 8=[8] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.256127+0000", 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.440474637999998, 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019700597, 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 282 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256127+0000", 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256127+0000", 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256128+0000", 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3030000000000001e-06 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256132+0000", 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9990000000000002e-06 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256143+0000", 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1375e-05 2024-09-15T01:33:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256147+0000", 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1949999999999996e-06 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256180+0000", 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2387e-05 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256244+0000", 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4369000000000007e-05 2024-09-15T01:33:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256731+0000", 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00048626099999999998 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256921+0000", 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000190743 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256928+0000", 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8510000000000001e-06 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256963+0000", 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5314000000000001e-05 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.256979+0000", 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5174000000000001e-05 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.258442+0000", 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00146372 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.275827+0000", 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017384905999999999 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=85, reqid=osd.11.0:84, at_version=0'0, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.257015+0000", 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.439586744000003, 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019035946000000002, 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257015+0000", 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257015+0000", 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257015+0000", 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7100000000000002e-07 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257025+0000", 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0115999999999999e-05 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257047+0000", 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1645e-05 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257051+0000", 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4139999999999996e-06 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257094+0000", 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2843999999999998e-05 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257104+0000", 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.798e-06 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257241+0000", 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013668799999999999 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257248+0000", 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5299999999999999e-06 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259196+0000", 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019477850000000001 2024-09-15T01:33:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259787+0000", 2024-09-15T01:33:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00059073299999999997 2024-09-15T01:33:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.276050+0000", 2024-09-15T01:33:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016263822000000001 2024-09-15T01:33:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 44/31 ECSubWrite(tid=30, reqid=osd.8.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.257430+0000", 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.439170783999998, 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018688996999999999, 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257430+0000", 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257430+0000", 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257431+0000", 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2499999999999995e-07 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257443+0000", 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1728e-05 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257465+0000", 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2013000000000001e-05 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257469+0000", 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6750000000000001e-06 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257498+0000", 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8670000000000002e-05 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257503+0000", 2024-09-15T01:33:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1019999999999996e-06 2024-09-15T01:33:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257572+0000", 2024-09-15T01:33:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8450999999999999e-05 2024-09-15T01:33:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259198+0000", 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001626239 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259263+0000", 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5491e-05 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259288+0000", 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4766e-05 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.276119+0000", 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016831336999999998 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=86, reqid=osd.11.0:86, at_version=44'42, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.257959+0000", 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.438642469999998, 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018797019000000002, 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257959+0000", 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257959+0000", 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257959+0000", 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257972+0000", 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.261e-05 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257993+0000", 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1067e-05 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.257996+0000", 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4690000000000002e-06 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.258029+0000", 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2691999999999998e-05 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.258035+0000", 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0619999999999997e-06 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259524+0000", 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001488781 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259533+0000", 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3190000000000001e-06 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259567+0000", 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3856999999999998e-05 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259896+0000", 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032855100000000001 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.276756+0000", 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016860199999999999 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:46.259034+0000", 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.437567512000001, 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018774057, 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259034+0000", 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259034+0000", 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259036+0000", 2024-09-15T01:33:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0829999999999998e-06 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259049+0000", 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3283e-05 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259070+0000", 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0488000000000001e-05 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259073+0000", 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5310000000000002e-06 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259569+0000", 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049599699999999997 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259578+0000", 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7709999999999995e-06 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259766+0000", 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000187954 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259774+0000", 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1920000000000005e-06 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259800+0000", 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.635e-05 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.259910+0000", 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109957 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:46.277808+0000", 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017897451000000002 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 47/31 ECSubWriteReply(tid=162, last_complete=47'9, committed=1, applied=1))", 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:49.375678+0000", 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.320923684999997, 2024-09-15T01:33:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.71735708600000003, 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.375678+0000", 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.375678+0000", 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.375678+0000", 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2800000000000003e-07 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:49.375680+0000", 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.635e-06 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.092521+0000", 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.716841324 2024-09-15T01:33:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.092538+0000", 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6983e-05 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.092633+0000", 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5618999999999994e-05 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:50.093035+0000", 2024-09-15T01:33:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000401197 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:509 3.2s0 3:4568dce4:::smithi17529310-45:head [copy-from ver 70] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.073110+0000", 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.623491088000002, 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.92744239399999995, 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 509 2024-09-15T01:33:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073110+0000", 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073110+0000", 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073112+0000", 2024-09-15T01:33:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073120+0000", 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8269999999999998e-06 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073134+0000", 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3934000000000001e-05 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073142+0000", 2024-09-15T01:33:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9899999999999997e-06 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073519+0000", 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037723500000000002 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073522+0000", 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9179999999999998e-06 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.985732+0000", 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.91221027799999999 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.985887+0000", 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015458799999999999 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.997489+0000", 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011601800000000001 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.997763+0000", 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000274586 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.997775+0000", 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1488000000000001e-05 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.998089+0000", 2024-09-15T01:33:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031444300000000003 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.998169+0000", 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9367000000000001e-05 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.000501+0000", 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023319030000000002 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:58.000553+0000", 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1966999999999999e-05 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:510 3.2s0 3:4568dce4:::smithi17529310-45:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:32:57.073150+0000", 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.623451699999997, 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.92501677100000002, 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 510 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073150+0000", 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073150+0000", 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073150+0000", 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6300000000000005e-07 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073157+0000", 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5490000000000003e-06 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073165+0000", 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.926e-06 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073169+0000", 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4429999999999996e-06 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073547+0000", 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037755599999999997 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.073549+0000", 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.616e-06 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.986765+0000", 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.91321570200000002 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.986800+0000", 2024-09-15T01:33:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5169999999999997e-05 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.997953+0000", 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011152998000000001 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.998084+0000", 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013044400000000001 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:32:57.998166+0000", 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2804000000000002e-05 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s6 63/31 ECSubWrite(tid=46, reqid=client.4477.0:596, at_version=0'0, trim_to=0'0, roll_forward_to=63'33))", 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:05.101346+0000", 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 40.595255303000002, 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.96745142500000003, 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:05.101346+0000", 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:05.101346+0000", 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:05.101347+0000", 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:05.101358+0000", 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0951000000000001e-05 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068502+0000", 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.96714420499999998 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068510+0000", 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1589999999999993e-06 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068576+0000", 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5767999999999998e-05 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068586+0000", 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-06 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068671+0000", 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5958000000000006e-05 2024-09-15T01:33:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068684+0000", 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2228999999999999e-05 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068724+0000", 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0299999999999997e-05 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068738+0000", 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3711e-05 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:06.068797+0000", 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9648000000000002e-05 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s1 70/69 ECSubWrite(tid=72, reqid=client.4477.0:636, at_version=0'0, trim_to=0'0, roll_forward_to=70'43))", 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:12.107237+0000", 2024-09-15T01:33:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.589364271999997, 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.66967561099999995, 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.107237+0000", 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.107237+0000", 2024-09-15T01:33:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.107238+0000", 2024-09-15T01:33:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5700000000000002e-07 2024-09-15T01:33:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.107247+0000", 2024-09-15T01:33:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4469999999999995e-06 2024-09-15T01:33:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776489+0000", 2024-09-15T01:33:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.66924175600000002 2024-09-15T01:33:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776495+0000", 2024-09-15T01:33:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0680000000000002e-06 2024-09-15T01:33:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776557+0000", 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2080000000000002e-05 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776565+0000", 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5859999999999999e-06 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776657+0000", 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2614000000000005e-05 2024-09-15T01:33:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776666+0000", 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1859999999999998e-06 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776711+0000", 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4829000000000001e-05 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776752+0000", 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0345000000000001e-05 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:12.776913+0000", 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016104300000000001 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:648 3.3s0 3:c07cd22b:::smithi17529310-3:head [copy-from ver 121] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.135954+0000", 2024-09-15T01:33:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.560647416999998, 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019156803, 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 648 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.135954+0000", 2024-09-15T01:33:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.135954+0000", 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.135955+0000", 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4169999999999999e-06 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.135960+0000", 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4530000000000004e-06 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.135973+0000", 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2955999999999999e-05 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.135977+0000", 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2300000000000002e-06 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.136044+0000", 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7218999999999995e-05 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.136111+0000", 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7107999999999995e-05 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.149703+0000", 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013591486999999999 2024-09-15T01:33:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.150702+0000", 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00099899199999999998 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.150713+0000", 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1345999999999999e-05 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.151097+0000", 2024-09-15T01:33:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038442100000000002 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.151283+0000", 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018522300000000001 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.155064+0000", 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037815850000000001 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.155111+0000", 2024-09-15T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6366000000000003e-05 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:694 3.3s0 3:c4339fa1:::smithi17529310-16:head [copy-from ver 53] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.298369+0000", 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.398232313000001, 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030068248999999998, 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 694 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298369+0000", 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298369+0000", 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298371+0000", 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7090000000000001e-06 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298377+0000", 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6710000000000002e-06 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298388+0000", 2024-09-15T01:33:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158000000000001e-05 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298394+0000", 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2959999999999998e-06 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298426+0000", 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2011000000000002e-05 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:45.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298502+0000", 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6416999999999997e-05 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.321468+0000", 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022965877999999999 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.321958+0000", 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049022300000000003 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.321967+0000", 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6989999999999992e-06 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.323335+0000", 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013682259999999999 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.323352+0000", 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6782999999999998e-05 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.328391+0000", 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005038832 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.328437+0000", 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6346e-05 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:695 3.3s0 3:c4339fa1:::smithi17529310-16:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:13.298402+0000", 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.398199150000003, 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024018102, 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 695 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298402+0000", 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298402+0000", 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298403+0000", 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298405+0000", 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8169999999999999e-06 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298409+0000", 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4220000000000002e-06 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298411+0000", 2024-09-15T01:33:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298623+0000", 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000211516 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.298679+0000", 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5652999999999997e-05 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.322176+0000", 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023497759 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.322288+0000", 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111217 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:13.322420+0000", 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013263399999999999 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:815 3.3s0 3:c07cd22b:::smithi17529310-3:2b [read 0~3620864,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:27.099958+0000", 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.596642810999999, 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021511853000000001, 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 815 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.099958+0000", 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.099958+0000", 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.099959+0000", 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3300000000000003e-07 2024-09-15T01:33:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.099963+0000", 2024-09-15T01:33:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6370000000000001e-06 2024-09-15T01:33:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.099969+0000", 2024-09-15T01:33:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9700000000000002e-06 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.099974+0000", 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7210000000000002e-06 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.100063+0000", 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8450999999999997e-05 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.100067+0000", 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.566e-06 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.101799+0000", 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017316199999999999 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.101894+0000", 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5062000000000006e-05 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.121470+0000", 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019576393000000001 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:816 3.3s0 3:c07cd22b:::smithi17529310-3:2b [read 0~3620864,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:27.099996+0000", 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.596605175000001, 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023672441999999998, 2024-09-15T01:33:45.797 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 816 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.099996+0000", 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.099996+0000", 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.099997+0000", 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2099999999999997e-07 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.100001+0000", 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1099999999999996e-06 2024-09-15T01:33:45.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.100007+0000", 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9440000000000002e-06 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.100011+0000", 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1019999999999999e-06 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.100095+0000", 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3856000000000005e-05 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.100102+0000", 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.905e-06 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.105555+0000", 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054537079999999998 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.105671+0000", 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011579899999999999 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:27.123669+0000", 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017997497000000001 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:847 3.2s0 3:4568dce4:::smithi17529310-45:head [rollback 2c] snapc 32=[32,30,2e,2c,2b,29,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:45.802 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:28.132782+0000", 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.563819578, 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018276447000000001, 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 847 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132782+0000", 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132782+0000", 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132782+0000", 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3299999999999998e-07 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132784+0000", 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.119e-06 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132789+0000", 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1150000000000002e-06 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132792+0000", 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0900000000000001e-06 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.133938+0000", 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011458390000000001 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.134006+0000", 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7887999999999998e-05 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.134454+0000", 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044790299999999998 2024-09-15T01:33:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.134533+0000", 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9054000000000006e-05 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.135751+0000", 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012183140000000001 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.135790+0000", 2024-09-15T01:33:45.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8844999999999998e-05 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.135938+0000", 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014823399999999999 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.136294+0000", 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035533100000000002 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.151058+0000", 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014764382 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4477.0:848 3.2s0 3:4568dce4:::smithi17529310-45:head [append 0~0] snapc 32=[32,30,2e,2c,2b,29,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-15T01:33:28.132814+0000", 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.563787388000001, 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018603701, 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-15T01:33:45.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4477", 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 848 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132814+0000", 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132814+0000", 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-15T01:33:45.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132814+0000", 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000002e-07 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132819+0000", 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.916e-06 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132842+0000", 2024-09-15T01:33:45.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3249999999999999e-05 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.132847+0000", 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7770000000000002e-06 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.134625+0000", 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001777657 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.134726+0000", 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010100899999999999 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.134992+0000", 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000265954 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.135128+0000", 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000135828 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.135752+0000", 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062419799999999996 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.135791+0000", 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8902000000000001e-05 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.135941+0000", 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015005699999999999 2024-09-15T01:33:45.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.136546+0000", 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00060493799999999996 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-15T01:33:28.151418+0000", 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014871749 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-15T01:33:45.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-15T01:33:45.816 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-15T01:33:45.817 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:33:45.926 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:45.927 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:45.936 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:33:46.035 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:46.035 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:46.035 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:46.036 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:46.036 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:46.044 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:33:46.136 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-15T01:33:46.146 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:46.146 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:46.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:46.146 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:46.146 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.146 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:46.146 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:33:46.146 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.896287843000003, 2024-09-15T01:33:46.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019581815999999998, 2024-09-15T01:33:46.146 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246968+0000", 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246969+0000", 2024-09-15T01:33:46.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7000000000000003e-07 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247008+0000", 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9354999999999998e-05 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247051+0000", 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2849000000000003e-05 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247060+0000", 2024-09-15T01:33:46.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4589999999999998e-06 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248680+0000", 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001620058 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248690+0000", 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8660000000000001e-06 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248945+0000", 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025509900000000001 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251105+0000", 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021604290000000002 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251177+0000", 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1680999999999994e-05 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251189+0000", 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2661e-05 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266550+0000", 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015360389 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=28, reqid=osd.8.0:28, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:46.151 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.893105972000001, 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022199446000000001, 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250150+0000", 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4700000000000001e-07 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250167+0000", 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7384000000000002e-05 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250202+0000", 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4912e-05 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250210+0000", 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6029999999999999e-06 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250258+0000", 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8109999999999998e-05 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250265+0000", 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4400000000000002e-06 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250466+0000", 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000201227 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251113+0000", 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00064769299999999997 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251179+0000", 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5895999999999994e-05 2024-09-15T01:33:46.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251196+0000", 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6810999999999999e-05 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272349+0000", 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021153222999999999 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.156 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.891751843999998, 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023186075, 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251504+0000", 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251505+0000", 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.09e-07 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251531+0000", 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5834999999999999e-05 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251574+0000", 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2879000000000001e-05 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251580+0000", 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8460000000000002e-06 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251639+0000", 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8331000000000001e-05 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251649+0000", 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9930000000000007e-06 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251890+0000", 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00024173400000000001 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251923+0000", 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2607999999999997e-05 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251985+0000", 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2238999999999998e-05 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251998+0000", 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2979999999999999e-05 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274690+0000", 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022691821000000001 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.161 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s1 44/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=44'26))", 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.890534846000001, 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019696631999999999, 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252721+0000", 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252722+0000", 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3099999999999996e-07 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252734+0000", 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1576999999999999e-05 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252780+0000", 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6820999999999998e-05 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252788+0000", 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2710000000000003e-06 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252848+0000", 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0718999999999999e-05 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252859+0000", 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0342e-05 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252960+0000", 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000101741 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252970+0000", 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0008e-05 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253967+0000", 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00099612599999999996 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253987+0000", 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0064e-05 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018431032 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.166 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=154, reqid=client.4477.0:282, at_version=44'19, trim_to=0'0, roll_forward_to=44'17))", 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.886523185999998, 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023652260000000001, 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256733+0000", 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256734+0000", 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3900000000000004e-07 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.168 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256754+0000", 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0279e-05 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256785+0000", 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0703000000000001e-05 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256798+0000", 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3606e-05 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.169 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256843+0000", 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5065000000000003e-05 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.256851+0000", 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5580000000000003e-06 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257013+0000", 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016264 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257339+0000", 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032564200000000001 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257402+0000", 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3162000000000003e-05 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258083+0000", 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00068126099999999995 2024-09-15T01:33:46.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280385+0000", 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022301504999999999 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=85, reqid=osd.11.0:84, at_version=0'0, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.886199402000003, 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020078747000000001, 2024-09-15T01:33:46.172 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257056+0000", 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.173 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257057+0000", 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5000000000000002e-07 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257067+0000", 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5540000000000003e-06 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257097+0000", 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9892000000000001e-05 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.174 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257102+0000", 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3360000000000002e-06 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257155+0000", 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3313000000000003e-05 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257164+0000", 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1060000000000006e-06 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.175 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257257+0000", 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2678999999999998e-05 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257341+0000", 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3459000000000002e-05 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257403+0000", 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2477000000000005e-05 2024-09-15T01:33:46.176 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257420+0000", 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7170999999999999e-05 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277135+0000", 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019715010000000002 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.177 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=30, reqid=osd.8.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.885932130999997, 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020096306000000001, 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:46.178 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257324+0000", 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3099999999999997e-07 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257335+0000", 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0727000000000001e-05 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257371+0000", 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5583999999999997e-05 2024-09-15T01:33:46.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257378+0000", 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.754e-06 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257427+0000", 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9048999999999997e-05 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257475+0000", 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8140999999999999e-05 2024-09-15T01:33:46.180 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257605+0000", 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012981700000000001 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257640+0000", 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.557e-05 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257674+0000", 2024-09-15T01:33:46.181 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4215999999999999e-05 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257692+0000", 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7988000000000001e-05 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277420+0000", 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019727728999999999 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.182 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=86, reqid=osd.11.0:86, at_version=44'42, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.885340876999997, 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022807320999999998, 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.183 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257915+0000", 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257916+0000", 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257951+0000", 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5754999999999999e-05 2024-09-15T01:33:46.184 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257982+0000", 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0732000000000003e-05 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257987+0000", 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1900000000000003e-06 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258036+0000", 2024-09-15T01:33:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8547999999999997e-05 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258047+0000", 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1082e-05 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258512+0000", 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00046539599999999997 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258527+0000", 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5121e-05 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258568+0000", 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0305000000000001e-05 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258581+0000", 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3434999999999999e-05 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280722+0000", 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022141137000000002 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 44/31 ECSubWrite(tid=155, reqid=client.4477.0:282, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.884416080000001, 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021774615000000001, 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258840+0000", 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258841+0000", 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5899999999999995e-07 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258853+0000", 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2115e-05 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258889+0000", 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6007000000000003e-05 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258896+0000", 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.190 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258958+0000", 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2242999999999993e-05 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258966+0000", 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2370000000000003e-06 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259070+0000", 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000104171 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.191 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259104+0000", 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4569000000000001e-05 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259140+0000", 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5698000000000003e-05 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259155+0000", 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4541e-05 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.192 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280614+0000", 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021459865000000002 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.883986704999998, 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027513342999999999, 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259269+0000", 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259270+0000", 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.33e-07 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.194 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259295+0000", 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4413000000000002e-05 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259335+0000", 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0154999999999998e-05 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259344+0000", 2024-09-15T01:33:46.195 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8890000000000008e-06 2024-09-15T01:33:46.196 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.196 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.196 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.196 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259414+0000", 2024-09-15T01:33:46.196 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9929000000000001e-05 2024-09-15T01:33:46.196 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.196 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.196 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.196 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259423+0000", 2024-09-15T01:33:46.196 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0289999999999994e-06 2024-09-15T01:33:46.196 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263376+0000", 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003953036 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263393+0000", 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7444000000000001e-05 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263481+0000", 2024-09-15T01:33:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8251999999999995e-05 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266784+0000", 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003303074 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286783+0000", 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019998189 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.198 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=31, reqid=client.4477.0:288, at_version=44'22, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.879219333999998, 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022720738000000001, 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:46.199 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264037+0000", 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6599999999999995e-07 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264398+0000", 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00036057299999999999 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264436+0000", 2024-09-15T01:33:46.200 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7968e-05 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264444+0000", 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8949999999999998e-06 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264489+0000", 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4954999999999999e-05 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.264499+0000", 2024-09-15T01:33:46.201 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0574000000000001e-05 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266178+0000", 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001678543 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.266188+0000", 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0624e-05 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.202 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268728+0000", 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025397240000000001 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268912+0000", 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018415400000000001 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286757+0000", 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.017844961999999999 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.203 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=32, reqid=client.4477.0:289, at_version=44'23, trim_to=0'0, roll_forward_to=44'20))", 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.876019896000003, 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019719520000000001, 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:33:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267236+0000", 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267237+0000", 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3099999999999998e-07 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.205 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267495+0000", 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00025857700000000001 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267528+0000", 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2617999999999998e-05 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267534+0000", 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0159999999999997e-06 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267584+0000", 2024-09-15T01:33:46.206 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0176999999999997e-05 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267594+0000", 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0161e-05 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267768+0000", 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017420000000000001 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268732+0000", 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00096413399999999995 2024-09-15T01:33:46.207 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268818+0000", 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5583999999999999e-05 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268943+0000", 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012452399999999999 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286956+0000", 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018012997999999999 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.208 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs1 44/31 ECSubWrite(tid=91, reqid=osd.11.0:87, at_version=0'0, trim_to=0'0, roll_forward_to=44'44))", 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.875927451999999, 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019815672999999999, 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.209 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267328+0000", 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267329+0000", 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7500000000000001e-07 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267339+0000", 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0351e-05 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.210 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267385+0000", 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6008999999999999e-05 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267393+0000", 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7810000000000005e-06 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267443+0000", 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0139000000000002e-05 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267454+0000", 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1012999999999999e-05 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267585+0000", 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013078699999999999 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268732+0000", 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001146768 2024-09-15T01:33:46.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268817+0000", 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5724999999999997e-05 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268838+0000", 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0954e-05 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.287144+0000", 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018305770999999998 2024-09-15T01:33:46.213 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=32, reqid=client.4477.0:294, at_version=44'23, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.872953238999997, 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021391184000000001, 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.214 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270303+0000", 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270304+0000", 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7499999999999998e-07 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270326+0000", 2024-09-15T01:33:46.215 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2821999999999999e-05 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270366+0000", 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9708e-05 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270374+0000", 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6709999999999999e-06 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270433+0000", 2024-09-15T01:33:46.216 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9274999999999997e-05 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270444+0000", 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.131e-05 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271974+0000", 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001529996 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271986+0000", 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1443e-05 2024-09-15T01:33:46.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272180+0000", 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019405299999999999 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272366+0000", 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018627699999999999 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.291694+0000", 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019327654 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.218 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.872356560999997, 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024535670999999998, 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.219 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270899+0000", 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270900+0000", 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.23e-07 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270920+0000", 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.022e-05 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.220 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270953+0000", 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3136000000000003e-05 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270960+0000", 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4280000000000001e-06 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272041+0000", 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010816440000000001 2024-09-15T01:33:46.221 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272050+0000", 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5269999999999998e-06 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272149+0000", 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9464999999999997e-05 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.222 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272182+0000", 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2792e-05 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272209+0000", 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6409000000000001e-05 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272435+0000", 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000226691 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295435+0000", 2024-09-15T01:33:46.223 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022999636 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 44/31 ECSubWrite(tid=34, reqid=client.4477.0:296, at_version=44'25, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.871790453999999, 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024251137999999998, 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.224 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271465+0000", 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271466+0000", 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8899999999999998e-07 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.225 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271488+0000", 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1875999999999999e-05 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271522+0000", 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4041999999999997e-05 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271529+0000", 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4620000000000001e-06 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272189+0000", 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066068699999999997 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272198+0000", 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2900000000000002e-06 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272284+0000", 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6638999999999995e-05 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.227 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272322+0000", 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7125000000000001e-05 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272357+0000", 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5626000000000001e-05 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272458+0000", 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000100343 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.228 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295717+0000", 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023259058999999999 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s1 44/31 ECSubWrite(tid=35, reqid=client.4477.0:291, at_version=0'0, trim_to=0'0, roll_forward_to=44'25))", 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 59.870183316999999, 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022393413000000001, 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.229 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.230 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:46.230 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.230 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.230 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.230 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:46.230 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.230 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.230 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.230 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273073+0000", 2024-09-15T01:33:46.230 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0999999999999998e-07 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273083+0000", 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0015e-05 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273110+0000", 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6798999999999999e-05 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273117+0000", 2024-09-15T01:33:46.231 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5379999999999999e-06 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273155+0000", 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8563999999999999e-05 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273163+0000", 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4939999999999997e-06 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.232 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273248+0000", 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.5290000000000005e-05 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273254+0000", 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3149999999999997e-06 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273288+0000", 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3958000000000003e-05 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.233 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273303+0000", 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4731e-05 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.295466+0000", 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022162998999999999 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s2 54/31 ECSubWrite(tid=63, reqid=client.4477.0:459, at_version=54'49, trim_to=0'0, roll_forward_to=54'46))", 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.094073+0000", 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 50.049182426999998, 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.82711023299999997, 2024-09-15T01:33:46.234 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094073+0000", 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094073+0000", 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.235 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094074+0000", 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4099999999999999e-07 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.094105+0000", 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1087000000000002e-05 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920807+0000", 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.82670179300000002 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.236 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920813+0000", 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8100000000000003e-06 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920878+0000", 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4870999999999996e-05 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920885+0000", 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3860000000000001e-06 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.237 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920975+0000", 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9797999999999996e-05 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.920981+0000", 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2419999999999997e-06 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.921033+0000", 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1811999999999998e-05 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.921048+0000", 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5135e-05 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.921184+0000", 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013575800000000001 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpRead(3.fs1 69/31 ECSubRead(tid=220, to_read={3:f73c1a72:::smithi17529310-29:head=0,4096,0}, subchunks={3:f73c1a72:::smithi17529310-29:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:11.104731+0000", 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 35.038525165000003, 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.77828088100000004, 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.104731+0000", 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.240 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.104731+0000", 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.104731+0000", 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8699999999999999e-07 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.104736+0000", 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7659999999999998e-06 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.882809+0000", 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.77807284600000004 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.882814+0000", 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8889999999999999e-06 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.882901+0000", 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6193000000000006e-05 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:11.883012+0000", 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000111 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.3s4 98/31 ECSubWrite(tid=336, reqid=client.4477.0:954, at_version=0'0, trim_to=0'0, roll_forward_to=98'95))", 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:40.564292+0000", 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 5.5789643179999997, 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.51735805700000004, 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.243 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:40.564292+0000", 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:40.564292+0000", 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:40.564292+0000", 2024-09-15T01:33:46.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3799999999999998e-07 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:40.564298+0000", 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4540000000000004e-06 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:41.081327+0000", 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.51702875100000001 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:41.081334+0000", 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0790000000000003e-06 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:41.081376+0000", 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2011000000000001e-05 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:41.081385+0000", 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6780000000000006e-06 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:41.081505+0000", 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119733 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:41.081522+0000", 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7719000000000001e-05 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:41.081561+0000", 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8129e-05 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:41.081580+0000", 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9877e-05 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:41.081650+0000", 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9188000000000002e-05 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.248 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:46.250 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:33:46.250 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:33:46.349 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:46.349 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:46.349 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:46.349 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:46.357 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:33:46.455 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:46.455 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:46.455 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:46.455 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:46.455 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:46.463 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:33:46.561 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:46.561 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:46.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:46.561 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:46.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.561 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:46.561 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:46.561 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.317301127, 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022664864999999999, 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.240988+0000", 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.16e-07 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241155+0000", 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016715400000000001 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241334+0000", 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017883600000000001 2024-09-15T01:33:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241342+0000", 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1519999999999992e-06 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241446+0000", 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010419 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241468+0000", 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1747e-05 2024-09-15T01:33:46.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.244959+0000", 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0034911600000000001 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.244970+0000", 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1131000000000001e-05 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245013+0000", 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3214000000000003e-05 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245498+0000", 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00048501499999999999 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263652+0000", 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018154050000000001 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=80, reqid=osd.11.0:80, at_version=44'28, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.317103166999999, 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022656372000000001, 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:46.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241186+0000", 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9399999999999999e-07 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241203+0000", 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7152000000000001e-05 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241707+0000", 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050421299999999999 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241714+0000", 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4409999999999998e-06 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241773+0000", 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9762000000000001e-05 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.241783+0000", 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.6860000000000001e-06 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245035+0000", 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032521130000000001 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245184+0000", 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014925800000000001 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245769+0000", 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058491399999999996 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246140+0000", 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037060199999999999 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.263842+0000", 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.017702037 2024-09-15T01:33:46.571 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=81, reqid=osd.11.0:81, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.315886049, 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034719353000000001, 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242403+0000", 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1600000000000001e-07 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242427+0000", 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4022000000000001e-05 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242460+0000", 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2496000000000002e-05 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.242476+0000", 2024-09-15T01:33:46.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6503000000000001e-05 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245083+0000", 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.002606531 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245092+0000", 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7679999999999997e-06 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245298+0000", 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000205971 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245772+0000", 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00047452200000000001 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245817+0000", 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4474999999999997e-05 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246171+0000", 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035433200000000001 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277122+0000", 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030951117 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.315262517999997, 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031640535999999997, 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243026+0000", 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.49e-07 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243044+0000", 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7946999999999999e-05 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243140+0000", 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5311999999999998e-05 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.243147+0000", 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2069999999999998e-06 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245010+0000", 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018629230000000001 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245020+0000", 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0774999999999999e-05 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245472+0000", 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045200900000000002 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245773+0000", 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00030029999999999998 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.245817+0000", 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4367999999999999e-05 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246317+0000", 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050021500000000003 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274667+0000", 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028349330999999998 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.bs4 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.310998472000001, 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027771575999999999, 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247290+0000", 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247291+0000", 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7700000000000004e-07 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247302+0000", 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1246e-05 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247333+0000", 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1179000000000001e-05 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247341+0000", 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.571e-06 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247388+0000", 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6876999999999999e-05 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247395+0000", 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3359999999999997e-06 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247478+0000", 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2860000000000003e-05 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247487+0000", 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9029999999999993e-06 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247526+0000", 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8829999999999999e-05 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247546+0000", 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0551999999999999e-05 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275062+0000", 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027515444999999999 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as5 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.310775651999997, 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027915809999999999, 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247513+0000", 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247514+0000", 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9899999999999999e-07 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247524+0000", 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0396e-05 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247555+0000", 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0414000000000001e-05 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247564+0000", 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1900000000000001e-06 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247616+0000", 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1788e-05 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247628+0000", 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.288e-05 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247763+0000", 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013473700000000001 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247772+0000", 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3429999999999996e-06 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247804+0000", 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2533999999999997e-05 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247817+0000", 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2694000000000001e-05 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275429+0000", 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027612035 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.592 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s5 44/31 ECSubWrite(tid=65, reqid=osd.14.0:65, at_version=44'26, trim_to=0'0, roll_forward_to=44'24))", 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.308730306000001, 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.026123550999999998, 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.593 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249558+0000", 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249559+0000", 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3200000000000003e-07 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249584+0000", 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4284999999999999e-05 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.594 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249633+0000", 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9526e-05 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249641+0000", 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2239999999999995e-06 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249695+0000", 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3674999999999997e-05 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249705+0000", 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8300000000000008e-06 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251518+0000", 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018128409999999999 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251549+0000", 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0948999999999998e-05 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251591+0000", 2024-09-15T01:33:46.596 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2515999999999997e-05 2024-09-15T01:33:46.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.597 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.597 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.597 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251612+0000", 2024-09-15T01:33:46.597 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0494000000000001e-05 2024-09-15T01:33:46.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.597 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.597 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.597 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275682+0000", 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024070279 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.0s5 44/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=44'26))", 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.305536596000003, 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022959355000000001, 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.598 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252752+0000", 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252753+0000", 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7499999999999998e-07 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252764+0000", 2024-09-15T01:33:46.599 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1123e-05 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252899+0000", 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013444100000000001 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252906+0000", 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8650000000000001e-06 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252959+0000", 2024-09-15T01:33:46.600 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2698000000000003e-05 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252968+0000", 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2140000000000002e-06 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253058+0000", 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9926999999999995e-05 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253066+0000", 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6620000000000005e-06 2024-09-15T01:33:46.601 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253105+0000", 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8875999999999998e-05 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.253117+0000", 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2405000000000001e-05 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275711+0000", 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022594169000000001 2024-09-15T01:33:46.602 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=60, reqid=client.4477.0:306, at_version=44'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.287460195999998, 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024129333999999999, 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.603 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270828+0000", 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270829+0000", 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270845+0000", 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6013999999999999e-05 2024-09-15T01:33:46.604 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270871+0000", 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6140999999999999e-05 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270877+0000", 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7989999999999999e-06 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270947+0000", 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9480000000000006e-05 2024-09-15T01:33:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270959+0000", 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2395000000000001e-05 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274484+0000", 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.003524976 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274715+0000", 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000230595 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.606 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274806+0000", 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1130000000000003e-05 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274847+0000", 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1473000000000001e-05 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.294958+0000", 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020110711 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.607 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.287382186000002, 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020359914999999999, 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.608 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270906+0000", 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270907+0000", 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4200000000000001e-07 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270919+0000", 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1761e-05 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.609 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270936+0000", 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7761000000000002e-05 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270953+0000", 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6858e-05 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272042+0000", 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010882769999999999 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.610 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272051+0000", 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8400000000000007e-06 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272145+0000", 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3639000000000003e-05 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272153+0000", 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6240000000000002e-06 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272221+0000", 2024-09-15T01:33:46.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8281999999999994e-05 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272241+0000", 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9542e-05 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.291266+0000", 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.019025889000000001 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.285870940000002, 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.025858174000000001, 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:46.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272418+0000", 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272446+0000", 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7114000000000002e-05 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272489+0000", 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3711999999999998e-05 2024-09-15T01:33:46.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272498+0000", 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9169999999999995e-06 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274537+0000", 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0020384190000000001 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274545+0000", 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1999999999999994e-06 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274659+0000", 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011420499999999999 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274716+0000", 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7114999999999998e-05 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274807+0000", 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0346999999999994e-05 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274883+0000", 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.695e-05 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298276+0000", 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023392452000000001 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=62, reqid=client.4477.0:308, at_version=44'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:46.617 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.284823789999997, 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024943603000000002, 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.618 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273465+0000", 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.37e-07 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273478+0000", 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2768e-05 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273502+0000", 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3920000000000001e-05 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.619 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273508+0000", 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0909999999999998e-06 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274699+0000", 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011909240000000001 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274707+0000", 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5429999999999996e-06 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274788+0000", 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1804000000000005e-05 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274811+0000", 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2379e-05 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274902+0000", 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1031000000000003e-05 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274910+0000", 2024-09-15T01:33:46.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8359999999999992e-06 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298408+0000", 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023498769999999999 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=92, reqid=client.4477.0:309, at_version=44'31, trim_to=0'0, roll_forward_to=44'28))", 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.283724311999997, 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024021724000000001, 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.622 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274564+0000", 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274565+0000", 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9299999999999996e-07 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275016+0000", 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045075999999999997 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275077+0000", 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0582999999999999e-05 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275084+0000", 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1289999999999998e-06 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275144+0000", 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0566000000000003e-05 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275153+0000", 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4849999999999992e-06 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275302+0000", 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000149689 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.275327+0000", 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4337000000000001e-05 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277020+0000", 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001693153 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277074+0000", 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4122999999999997e-05 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298586+0000", 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021512006 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s6 44/31 ECSubWrite(tid=35, reqid=client.4477.0:296, at_version=0'0, trim_to=0'0, roll_forward_to=44'25))", 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.280182338000003, 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020797596000000002, 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278106+0000", 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278107+0000", 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8199999999999998e-07 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278119+0000", 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1567e-05 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278155+0000", 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6560999999999998e-05 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278163+0000", 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2819999999999999e-06 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278209+0000", 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6146999999999997e-05 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278219+0000", 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0699999999999999e-05 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278342+0000", 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000122061 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278350+0000", 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4460000000000001e-06 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280088+0000", 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0017376309999999999 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280107+0000", 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.986e-05 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.298904+0000", 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018796459000000001 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs4 44/31 ECSubWrite(tid=93, reqid=client.4477.0:310, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.279474335000003, 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.020226037999999998, 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.632 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278814+0000", 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.278815+0000", 2024-09-15T01:33:46.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6000000000000003e-07 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279204+0000", 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000388413 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279228+0000", 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4674000000000001e-05 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279233+0000", 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7849999999999999e-06 2024-09-15T01:33:46.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279272+0000", 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9502999999999998e-05 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279278+0000", 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0079999999999999e-06 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279401+0000", 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012238799999999999 2024-09-15T01:33:46.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280091+0000", 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00069018900000000002 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280213+0000", 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012227700000000001 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280231+0000", 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7408999999999999e-05 2024-09-15T01:33:46.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.299040+0000", 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.018809632 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.5s6 44/31 ECSubWrite(tid=15, reqid=client.4477.0:314, at_version=44'8, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.278924879000002, 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.026604415999999999, 2024-09-15T01:33:46.637 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279364+0000", 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279386+0000", 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.211e-05 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279421+0000", 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4523000000000001e-05 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279428+0000", 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4420000000000001e-06 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279474+0000", 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5995000000000003e-05 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279480+0000", 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.105e-06 2024-09-15T01:33:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.279645+0000", 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016430199999999999 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280092+0000", 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044707999999999998 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280214+0000", 2024-09-15T01:33:46.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012248399999999999 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280260+0000", 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5785000000000003e-05 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.305968+0000", 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.025708106000000001 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.642 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.5s6 44/31 ECSubWrite(tid=16, reqid=client.4477.0:314, at_version=0'0, trim_to=0'0, roll_forward_to=44'8))", 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.276270304999997, 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.024148095000000001, 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282018+0000", 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282019+0000", 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282028+0000", 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0969999999999995e-06 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282054+0000", 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5897e-05 2024-09-15T01:33:46.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282059+0000", 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7849999999999999e-06 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282101+0000", 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1801000000000002e-05 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282108+0000", 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0650000000000001e-06 2024-09-15T01:33:46.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282171+0000", 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2834000000000002e-05 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282844+0000", 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00067343600000000002 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282915+0000", 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1242999999999995e-05 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282933+0000", 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8136000000000001e-05 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306166+0000", 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023233052000000001 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 44/38 ECSubWrite(tid=63, reqid=client.4477.0:308, at_version=0'0, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:46.647 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.274686672999998, 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022584495, 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283602+0000", 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283603+0000", 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5899999999999995e-07 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283618+0000", 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5600999999999999e-05 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283648+0000", 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9295e-05 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283653+0000", 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1050000000000003e-06 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283703+0000", 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9883999999999999e-05 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283712+0000", 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3740000000000004e-06 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283859+0000", 2024-09-15T01:33:46.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000146959 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283869+0000", 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0295999999999999e-05 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283906+0000", 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6534e-05 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283922+0000", 2024-09-15T01:33:46.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6560000000000001e-05 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306187+0000", 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022264128000000001 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 47/38 ECSubWrite(tid=76, reqid=client.4477.0:376, at_version=47'25, trim_to=0'0, roll_forward_to=47'22))", 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 57.19448251, 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.61852187999999997, 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.652 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363806+0000", 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363807+0000", 2024-09-15T01:33:46.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5799999999999999e-07 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.363818+0000", 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1729000000000001e-05 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981818+0000", 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.61799953399999996 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981826+0000", 2024-09-15T01:33:46.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1049999999999995e-06 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981893+0000", 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7229000000000003e-05 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.981905+0000", 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1527e-05 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982064+0000", 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000159147 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982078+0000", 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982122+0000", 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.4354000000000003e-05 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982140+0000", 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8315e-05 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:49.982328+0000", 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018781200000000001 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s1 54/52 ECSubWrite(tid=119, reqid=client.4477.0:500, at_version=54'41, trim_to=0'0, roll_forward_to=54'38))", 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 50.346987478999999, 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.27782579699999999, 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.657 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211301+0000", 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211302+0000", 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.89e-07 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.211313+0000", 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1521e-05 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488807+0000", 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.27749379200000002 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488811+0000", 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2889999999999999e-06 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488867+0000", 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5322000000000003e-05 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488874+0000", 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9659999999999996e-06 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488953+0000", 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8591999999999996e-05 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488959+0000", 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0000000000000002e-06 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.488993+0000", 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4326999999999998e-05 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.489009+0000", 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6015000000000002e-05 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.489127+0000", 2024-09-15T01:33:46.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117584 2024-09-15T01:33:46.662 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.662 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.662 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.662 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.662 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.662 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:46.663 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:33:46.762 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:46.762 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:46.762 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:46.770 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:33:46.867 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:46.867 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:46.867 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:46.867 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:46.867 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:46.876 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:33:46.974 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:46.974 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.725044900999997, 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011534543, 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:46.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246893+0000", 2024-09-15T01:33:46.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246894+0000", 2024-09-15T01:33:46.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0500000000000002e-07 2024-09-15T01:33:46.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2585000000000001e-05 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246951+0000", 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4552999999999999e-05 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246961+0000", 2024-09-15T01:33:46.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0516000000000001e-05 2024-09-15T01:33:46.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247025+0000", 2024-09-15T01:33:46.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3540999999999994e-05 2024-09-15T01:33:46.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247039+0000", 2024-09-15T01:33:46.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4484000000000001e-05 2024-09-15T01:33:46.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247338+0000", 2024-09-15T01:33:46.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00029853699999999998 2024-09-15T01:33:46.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248324+0000", 2024-09-15T01:33:46.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0009866059999999999 2024-09-15T01:33:46.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248375+0000", 2024-09-15T01:33:46.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0337000000000003e-05 2024-09-15T01:33:46.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248387+0000", 2024-09-15T01:33:46.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2081e-05 2024-09-15T01:33:46.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.258427+0000", 2024-09-15T01:33:46.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010040498 2024-09-15T01:33:46.980 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.as6 44/38 ECSubWrite(tid=26, reqid=osd.3.0:25, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.724400762000002, 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010349413, 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:46.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247537+0000", 2024-09-15T01:33:46.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247538+0000", 2024-09-15T01:33:46.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0130000000000001e-06 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247552+0000", 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3723999999999999e-05 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247588+0000", 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5989999999999999e-05 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247595+0000", 2024-09-15T01:33:46.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5789999999999998e-06 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247648+0000", 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2757000000000003e-05 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247654+0000", 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3629999999999999e-06 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247737+0000", 2024-09-15T01:33:46.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2886000000000006e-05 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248326+0000", 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00058858900000000004 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248380+0000", 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3832999999999998e-05 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.248401+0000", 2024-09-15T01:33:46.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0843e-05 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.257886+0000", 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0094858359999999992 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 44/31 ECSubWrite(tid=87, reqid=osd.11.0:87, at_version=44'44, trim_to=0'0, roll_forward_to=44'40))", 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.712869408000003, 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015398044, 2024-09-15T01:33:46.986 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259068+0000", 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259069+0000", 2024-09-15T01:33:46.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2399999999999996e-07 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259095+0000", 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5527000000000001e-05 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259133+0000", 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8211999999999999e-05 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259140+0000", 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9330000000000004e-06 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259189+0000", 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.8797000000000001e-05 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259200+0000", 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0932e-05 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259425+0000", 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000225505 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.259453+0000", 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8152000000000001e-05 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.261606+0000", 2024-09-15T01:33:46.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021525870000000001 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.261621+0000", 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.4829e-05 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274466+0000", 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012845646 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 44/31 ECSubWrite(tid=39, reqid=client.4477.0:283, at_version=44'32, trim_to=0'0, roll_forward_to=44'30))", 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.703545198999997, 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014946894, 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268393+0000", 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0299999999999998e-07 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268593+0000", 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000199278 2024-09-15T01:33:46.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268633+0000", 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0367000000000002e-05 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268640+0000", 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.5100000000000001e-06 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:46.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268742+0000", 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010184899999999999 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.268752+0000", 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9299999999999998e-06 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269007+0000", 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000255016 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.269035+0000", 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8039999999999999e-05 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271592+0000", 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025566260000000002 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271610+0000", 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7629999999999999e-05 2024-09-15T01:33:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283339+0000", 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011729844999999999 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.6s6 44/38 ECSubWrite(tid=22, reqid=client.4477.0:293, at_version=44'14, trim_to=0'0, roll_forward_to=44'12))", 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 60.700761677999999, 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012212746, 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:46.997 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:46.998 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:46.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:46.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:46.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:46.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271176+0000", 2024-09-15T01:33:46.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:46.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:46.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271177+0000", 2024-09-15T01:33:46.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2200000000000003e-07 2024-09-15T01:33:46.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:46.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271394+0000", 2024-09-15T01:33:46.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021719400000000001 2024-09-15T01:33:46.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:46.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:46.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271421+0000", 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6842000000000002e-05 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271426+0000", 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9010000000000002e-06 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271464+0000", 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7928e-05 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271476+0000", 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1979e-05 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271800+0000", 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032464100000000002 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271832+0000", 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1501999999999998e-05 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271921+0000", 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9514999999999999e-05 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271940+0000", 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8436000000000001e-05 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.283389+0000", 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011449086000000001 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.002 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 55/31 ECSubWrite(tid=160, reqid=client.4477.0:512, at_version=55'53, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 49.896261004000003, 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011287287999999999, 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075677+0000", 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075698+0000", 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0384999999999999e-05 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075734+0000", 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6300999999999997e-05 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075743+0000", 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9549999999999998e-06 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075803+0000", 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0304999999999999e-05 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075813+0000", 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9539999999999999e-06 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075916+0000", 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102534 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075925+0000", 2024-09-15T01:33:47.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1509999999999993e-06 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075971+0000", 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6276999999999998e-05 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.075995+0000", 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3176999999999998e-05 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.086964+0000", 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010969504999999999 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 55/31 ECSubWrite(tid=161, reqid=client.4477.0:513, at_version=55'54, trim_to=0'0, roll_forward_to=51'50))", 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 49.895624560000002, 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010838802, 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.008 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076313+0000", 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076314+0000", 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076329+0000", 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5505000000000001e-05 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076356+0000", 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.6324000000000001e-05 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076362+0000", 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2820000000000002e-06 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076409+0000", 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7190000000000001e-05 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076419+0000", 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9450000000000005e-06 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076491+0000", 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1680999999999994e-05 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076497+0000", 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3810000000000004e-06 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076958+0000", 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00046042499999999998 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.076971+0000", 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2999999999999999e-05 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:57.087152+0000", 2024-09-15T01:33:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010181366000000001 2024-09-15T01:33:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 59/31 ECSubWrite(tid=179, reqid=client.4477.0:552, at_version=59'58, trim_to=0'0, roll_forward_to=56'56))", 2024-09-15T01:33:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 45.880455290999997, 2024-09-15T01:33:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010784815, 2024-09-15T01:33:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091482+0000", 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091483+0000", 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2999999999999999e-07 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091650+0000", 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016715 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091673+0000", 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3434999999999998e-05 2024-09-15T01:33:47.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091680+0000", 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7329999999999997e-06 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091721+0000", 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1088999999999999e-05 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091729+0000", 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9170000000000006e-06 2024-09-15T01:33:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.091940+0000", 2024-09-15T01:33:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021130000000000001 2024-09-15T01:33:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.092580+0000", 2024-09-15T01:33:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00063981200000000002 2024-09-15T01:33:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093157+0000", 2024-09-15T01:33:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057673800000000003 2024-09-15T01:33:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.093178+0000", 2024-09-15T01:33:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0911e-05 2024-09-15T01:33:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:01.102267+0000", 2024-09-15T01:33:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090895000000000004 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds3 66/31 ECSubWrite(tid=64, reqid=client.4477.0:609, at_version=66'56, trim_to=0'0, roll_forward_to=66'52))", 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:08.087598+0000", 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 38.884339783999998, 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010297796999999999, 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.087598+0000", 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.087598+0000", 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.087599+0000", 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1100000000000005e-07 2024-09-15T01:33:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088049+0000", 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00045032600000000002 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088087+0000", 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8138e-05 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088095+0000", 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.34e-06 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088153+0000", 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-05 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088165+0000", 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1386000000000001e-05 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088485+0000", 2024-09-15T01:33:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00032002700000000002 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088493+0000", 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8550000000000008e-06 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088596+0000", 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102278 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.088613+0000", 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7757e-05 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:08.097896+0000", 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0092822389999999994 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 71/31 ECSubWrite(tid=229, reqid=client.4477.0:651, at_version=71'127, trim_to=0'0, roll_forward_to=71'124))", 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.208012+0000", 2024-09-15T01:33:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 33.763925380000003, 2024-09-15T01:33:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.016322817, 2024-09-15T01:33:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208012+0000", 2024-09-15T01:33:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208012+0000", 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208013+0000", 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3099999999999997e-07 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208033+0000", 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9539e-05 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208064+0000", 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1705999999999998e-05 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208073+0000", 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3750000000000003e-06 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208136+0000", 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3736000000000001e-05 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208146+0000", 2024-09-15T01:33:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9029999999999999e-06 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208264+0000", 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117372 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208272+0000", 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.755e-06 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208300+0000", 2024-09-15T01:33:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7875999999999999e-05 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.208320+0000", 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9791000000000001e-05 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224335+0000", 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.016015033000000001 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 71/31 ECSubWrite(tid=231, reqid=client.4477.0:652, at_version=71'128, trim_to=0'0, roll_forward_to=71'124))", 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.209071+0000", 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 33.762866602999999, 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015677593, 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209071+0000", 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209071+0000", 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209072+0000", 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1399999999999997e-07 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209084+0000", 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1917e-05 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209402+0000", 2024-09-15T01:33:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00031803499999999999 2024-09-15T01:33:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209407+0000", 2024-09-15T01:33:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3179999999999997e-06 2024-09-15T01:33:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209454+0000", 2024-09-15T01:33:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6981000000000003e-05 2024-09-15T01:33:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209464+0000", 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0037999999999999e-05 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209592+0000", 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012783 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209599+0000", 2024-09-15T01:33:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9489999999999999e-06 2024-09-15T01:33:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209779+0000", 2024-09-15T01:33:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017987 2024-09-15T01:33:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.209797+0000", 2024-09-15T01:33:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.8071e-05 2024-09-15T01:33:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224749+0000", 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.01495207 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 71/31 ECSubWrite(tid=232, reqid=client.4477.0:652, at_version=0'0, trim_to=0'0, roll_forward_to=71'128))", 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 33.761457008000001, 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014733626, 2024-09-15T01:33:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210481+0000", 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8999999999999996e-07 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210491+0000", 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.7669999999999998e-06 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210530+0000", 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9039999999999999e-05 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210538+0000", 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2029999999999992e-06 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210601+0000", 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.3069000000000002e-05 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210612+0000", 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0372e-05 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210761+0000", 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000149044 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210833+0000", 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2084999999999998e-05 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210854+0000", 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.1424999999999999e-05 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210863+0000", 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1430000000000004e-06 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.225214+0000", 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014350887999999999 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=233, reqid=client.4477.0:653, at_version=0'0, trim_to=0'0, roll_forward_to=71'67))", 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 33.761443215, 2024-09-15T01:33:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013895739000000001, 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210495+0000", 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0599999999999999e-07 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210501+0000", 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2310000000000001e-06 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210562+0000", 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0612000000000002e-05 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210570+0000", 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4109999999999996e-06 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210619+0000", 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9419000000000002e-05 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210625+0000", 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5339999999999999e-06 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210691+0000", 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5905e-05 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210698+0000", 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0910000000000004e-06 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210831+0000", 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000133065 2024-09-15T01:33:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.210844+0000", 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3349e-05 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224390+0000", 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013545916 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=234, reqid=client.4477.0:654, at_version=71'68, trim_to=0'0, roll_forward_to=71'67))", 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 33.760292790999998, 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013353726, 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.211645+0000", 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2900000000000004e-07 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212326+0000", 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00068024600000000002 2024-09-15T01:33:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212354+0000", 2024-09-15T01:33:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8555999999999999e-05 2024-09-15T01:33:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212360+0000", 2024-09-15T01:33:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.755e-06 2024-09-15T01:33:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212397+0000", 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7086999999999999e-05 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212404+0000", 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7190000000000004e-06 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212510+0000", 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010600800000000001 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.212515+0000", 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3159999999999996e-06 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213020+0000", 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000504854 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213032+0000", 2024-09-15T01:33:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2048999999999999e-05 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.224999+0000", 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011966607000000001 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 71/31 ECSubWrite(tid=235, reqid=client.4477.0:654, at_version=0'0, trim_to=0'0, roll_forward_to=71'68))", 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 33.758385959000002, 2024-09-15T01:33:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011535082, 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213552+0000", 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4799999999999998e-07 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213560+0000", 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3100000000000003e-06 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213582+0000", 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2524e-05 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213588+0000", 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9009999999999999e-06 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213624+0000", 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5725000000000001e-05 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213629+0000", 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0679999999999996e-06 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213684+0000", 2024-09-15T01:33:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.4722999999999998e-05 2024-09-15T01:33:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.213690+0000", 2024-09-15T01:33:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4489999999999996e-06 2024-09-15T01:33:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.215563+0000", 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0018726680000000001 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.215576+0000", 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3021e-05 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:13.225087+0000", 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0095110449999999992 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.cs3 85/31 ECSubWrite(tid=351, reqid=osd.11.0:351, at_version=85'111, trim_to=0'0, roll_forward_to=85'107))", 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:27.141013+0000", 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 19.830925164, 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.109127261, 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.141013+0000", 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.141013+0000", 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.141013+0000", 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.06e-07 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.141037+0000", 2024-09-15T01:33:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4209000000000001e-05 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249578+0000", 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.10854032600000001 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249589+0000", 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1396000000000001e-05 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249676+0000", 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6737999999999995e-05 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249682+0000", 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0859999999999998e-06 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249864+0000", 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018231000000000001 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249872+0000", 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9370000000000004e-06 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249905+0000", 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2852000000000003e-05 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.249918+0000", 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2972e-05 2024-09-15T01:33:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:27.250140+0000", 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022172899999999999 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 88/31 ECSubWrite(tid=377, reqid=client.4477.0:877, at_version=88'202, trim_to=0'0, roll_forward_to=87'198))", 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:30.160526+0000", 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.811412180000001, 2024-09-15T01:33:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015647682999999999, 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160526+0000", 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160526+0000", 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160526+0000", 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4099999999999997e-07 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160546+0000", 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9592000000000001e-05 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160582+0000", 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5706e-05 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160588+0000", 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.7399999999999998e-06 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160647+0000", 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8125999999999999e-05 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160655+0000", 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4470000000000006e-06 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160917+0000", 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00026217299999999999 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.160924+0000", 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.2609999999999996e-06 2024-09-15T01:33:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163192+0000", 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022676559999999998 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163210+0000", 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7781000000000001e-05 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.176173+0000", 2024-09-15T01:33:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.01296336 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 88/31 ECSubWrite(tid=379, reqid=client.4477.0:878, at_version=88'203, trim_to=0'0, roll_forward_to=87'198))", 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:30.161562+0000", 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.810375490999999, 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.016881072, 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161562+0000", 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161562+0000", 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161563+0000", 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.1399999999999996e-07 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161579+0000", 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.641e-05 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161619+0000", 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.9190999999999998e-05 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161626+0000", 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6109999999999996e-06 2024-09-15T01:33:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161673+0000", 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6524999999999999e-05 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161682+0000", 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5799999999999998e-06 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.161833+0000", 2024-09-15T01:33:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015063199999999999 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163196+0000", 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0013626339999999999 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163354+0000", 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015873699999999999 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163372+0000", 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.7430000000000001e-05 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.178443+0000", 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015071508000000001 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 88/31 ECSubWrite(tid=381, reqid=client.4477.0:879, at_version=88'204, trim_to=0'0, roll_forward_to=88'199))", 2024-09-15T01:33:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:30.162496+0000", 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.80944135, 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.01606695, 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162496+0000", 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162496+0000", 2024-09-15T01:33:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162497+0000", 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.8599999999999997e-07 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162517+0000", 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9426e-05 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162545+0000", 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8138999999999999e-05 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162551+0000", 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0970000000000002e-06 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162598+0000", 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7187000000000001e-05 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162611+0000", 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2505e-05 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.162769+0000", 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015802899999999999 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163200+0000", 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043180900000000001 2024-09-15T01:33:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163359+0000", 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015842699999999999 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.163400+0000", 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.1702000000000002e-05 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.178563+0000", 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015162843000000001 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs5 88/31 ECSubWrite(tid=383, reqid=client.4477.0:879, at_version=0'0, trim_to=0'0, roll_forward_to=88'204))", 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:30.164585+0000", 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.807352271999999, 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014151115000000001, 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164585+0000", 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164585+0000", 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164586+0000", 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.3499999999999995e-07 2024-09-15T01:33:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164598+0000", 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1935e-05 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164624+0000", 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5931e-05 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164630+0000", 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6099999999999997e-06 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164685+0000", 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.5337999999999998e-05 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164691+0000", 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4999999999999996e-06 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164830+0000", 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013819900000000001 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.164840+0000", 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9820000000000003e-06 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.166597+0000", 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001757273 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.166633+0000", 2024-09-15T01:33:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.6386e-05 2024-09-15T01:33:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:30.178737+0000", 2024-09-15T01:33:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012103226 2024-09-15T01:33:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.084 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:47.086 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-15T01:33:47.140 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-15T01:33:47.183 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:47.183 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:47.184 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2024-09-15T01:33:47.184 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:47.192 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-15T01:33:47.289 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:47.290 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2024-09-15T01:33:47.290 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2024-09-15T01:33:47.290 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2024-09-15T01:33:47.290 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:47.298 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-15T01:33:47.397 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-15T01:33:47.397 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2024-09-15T01:33:47.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2024-09-15T01:33:47.397 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2024-09-15T01:33:47.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.397 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:74 3.0s0 3:0187269c:::smithi17529310-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:47.397 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 71.183973800999993, 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.071208814999999995, 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 74 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210617+0000", 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.1200000000000003e-07 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210620+0000", 2024-09-15T01:33:47.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.278e-06 2024-09-15T01:33:47.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210623+0000", 2024-09-15T01:33:47.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5779999999999999e-06 2024-09-15T01:33:47.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210626+0000", 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.295e-06 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211745+0000", 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011191770000000001 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.211880+0000", 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013497499999999999 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212353+0000", 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000473616 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212441+0000", 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.7268999999999994e-05 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212454+0000", 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3529999999999999e-05 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212483+0000", 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9224e-05 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212755+0000", 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00027191599999999998 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.281770+0000", 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.069014438999999997 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.281826+0000", 2024-09-15T01:33:47.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6005999999999999e-05 2024-09-15T01:33:47.404 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.404 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.404 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.404 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:75 3.0s0 3:0187269c:::smithi17529310-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:47.404 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:47.404 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 71.183941560999997, 2024-09-15T01:33:47.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.072451382999999994, 2024-09-15T01:33:47.404 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 75 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210649+0000", 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4700000000000002e-07 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210652+0000", 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9160000000000001e-06 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210657+0000", 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0529999999999997e-06 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.210660+0000", 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4339999999999999e-06 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212500+0000", 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001839732 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.212584+0000", 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.4759999999999995e-05 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.282036+0000", 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.069451322999999995 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.282131+0000", 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.577e-05 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.283100+0000", 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00096904800000000002 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds2 34/31 ECSubWrite(tid=16, reqid=client.4477.0:116, at_version=34'10, trim_to=0'0, roll_forward_to=34'8))", 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 71.134967111999998, 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.24562283800000001, 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259624+0000", 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9599999999999999e-07 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.259634+0000", 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0412e-05 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.504838+0000", 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.245203848 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.504852+0000", 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.3346999999999999e-05 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505000+0000", 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000148209 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505006+0000", 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6309999999999997e-06 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505068+0000", 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1962000000000001e-05 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505074+0000", 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0579999999999997e-06 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505103+0000", 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9828e-05 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505116+0000", 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2261e-05 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:36.505246+0000", 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000130786 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.ds2 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 61.147674465000001, 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033387655000000002, 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.414 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246916+0000", 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246917+0000", 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.61e-07 2024-09-15T01:33:47.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246937+0000", 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9741999999999998e-05 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246964+0000", 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7013999999999999e-05 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.246970+0000", 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8250000000000001e-06 2024-09-15T01:33:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247020+0000", 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0875999999999999e-05 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247032+0000", 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1280999999999999e-05 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249500+0000", 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0024682720000000001 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251106+0000", 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001606489 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251175+0000", 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8926999999999999e-05 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251258+0000", 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2441000000000006e-05 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280304+0000", 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029046026999999999 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs2 44/31 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=44'40, trim_to=0'0, roll_forward_to=40'34))", 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.247251+0000", 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 61.147339838999997, 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033110460000000001, 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.419 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247251+0000", 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247251+0000", 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247252+0000", 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9899999999999999e-07 2024-09-15T01:33:47.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247274+0000", 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.2160000000000001e-05 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247314+0000", 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0438000000000001e-05 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.247321+0000", 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.187e-06 2024-09-15T01:33:47.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249348+0000", 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.002026524 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249356+0000", 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7319999999999998e-06 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.249538+0000", 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000182239 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251107+0000", 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001569094 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251176+0000", 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.8707999999999993e-05 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251269+0000", 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3246999999999997e-05 2024-09-15T01:33:47.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280361+0000", 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029092331999999999 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s5 44/31 ECSubWrite(tid=28, reqid=osd.8.0:28, at_version=44'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.250215+0000", 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 61.144375668999999, 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033811606000000001, 2024-09-15T01:33:47.424 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250215+0000", 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250215+0000", 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250216+0000", 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3200000000000003e-07 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250240+0000", 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3849000000000002e-05 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250277+0000", 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7469999999999999e-05 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250284+0000", 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0979999999999996e-06 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250325+0000", 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.0184e-05 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.250331+0000", 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.5060000000000001e-06 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254385+0000", 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0040539560000000001 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254397+0000", 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2502e-05 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254432+0000", 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.4631000000000001e-05 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254987+0000", 2024-09-15T01:33:47.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000555011 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.284027+0000", 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029039466999999999 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.4s5 44/31 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=44'20, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.251962+0000", 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 61.142628354000003, 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034678124999999997, 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.429 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251962+0000", 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251962+0000", 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251963+0000", 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4799999999999998e-07 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.251986+0000", 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3221e-05 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252099+0000", 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112687 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.252107+0000", 2024-09-15T01:33:47.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0399999999999993e-06 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254434+0000", 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0023271300000000002 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254447+0000", 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2607e-05 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254678+0000", 2024-09-15T01:33:47.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000230869 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254847+0000", 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00016962300000000001 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.254891+0000", 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.3441e-05 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.255014+0000", 2024-09-15T01:33:47.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012353200000000001 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.286640+0000", 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031626327000000003 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.fs2 44/31 ECSubWrite(tid=91, reqid=osd.11.0:87, at_version=0'0, trim_to=0'0, roll_forward_to=44'44))", 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.267434+0000", 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 61.127156694, 2024-09-15T01:33:47.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039301718999999999, 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267434+0000", 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267434+0000", 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267435+0000", 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.92e-07 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267445+0000", 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.8949999999999993e-06 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267477+0000", 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2768000000000002e-05 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.267485+0000", 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4499999999999998e-06 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281136+0000", 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.013651389999999999 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281144+0000", 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9549999999999992e-06 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281264+0000", 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012005799999999999 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282867+0000", 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00160271 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282917+0000", 2024-09-15T01:33:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9919000000000001e-05 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282948+0000", 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.1053000000000002e-05 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306736+0000", 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.023787729000000001 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s6 44/38 ECSubWrite(tid=60, reqid=client.4477.0:306, at_version=44'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270872+0000", 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 61.123718631999999, 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039161770999999998, 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270872+0000", 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270872+0000", 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270873+0000", 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0219999999999999e-06 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270902+0000", 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.9079000000000001e-05 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270938+0000", 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.5849999999999997e-05 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270958+0000", 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9762000000000002e-05 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280987+0000", 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010029733000000001 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.280996+0000", 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0280000000000006e-06 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281214+0000", 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00021798100000000001 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282866+0000", 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001651292 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282916+0000", 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.0145000000000002e-05 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282941+0000", 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.5066e-05 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.310034+0000", 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.027092813 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s4 44/31 ECSubWrite(tid=33, reqid=client.4477.0:295, at_version=44'24, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.270969+0000", 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 61.123621462999999, 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034685404000000003, 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270969+0000", 2024-09-15T01:33:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270969+0000", 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270970+0000", 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6199999999999998e-07 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.270986+0000", 2024-09-15T01:33:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6622000000000001e-05 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271019+0000", 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3077000000000002e-05 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271025+0000", 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8699999999999997e-06 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274545+0000", 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0035193350000000002 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274553+0000", 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.2810000000000008e-06 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274633+0000", 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0358000000000005e-05 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274827+0000", 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019425000000000001 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.276977+0000", 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021494420000000001 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.276994+0000", 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6733000000000001e-05 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.305655+0000", 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028660973999999999 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.8s4 44/31 ECSubWrite(tid=34, reqid=client.4477.0:296, at_version=44'25, trim_to=0'0, roll_forward_to=44'21))", 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 61.12310815, 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034642487, 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271483+0000", 2024-09-15T01:33:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2600000000000002e-07 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271500+0000", 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6745999999999999e-05 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271570+0000", 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.0242999999999998e-05 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.271578+0000", 2024-09-15T01:33:47.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0870000000000007e-06 2024-09-15T01:33:47.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274674+0000", 2024-09-15T01:33:47.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030959379999999999 2024-09-15T01:33:47.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274682+0000", 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9570000000000002e-06 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274766+0000", 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.3630999999999999e-05 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.274828+0000", 2024-09-15T01:33:47.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2609999999999999e-05 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.276978+0000", 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0021496520000000002 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.277024+0000", 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.5723999999999998e-05 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.306125+0000", 2024-09-15T01:33:47.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029101373 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s6 44/38 ECSubWrite(tid=61, reqid=client.4477.0:307, at_version=44'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.272445+0000", 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 61.122145332999999, 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041201330000000001, 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272445+0000", 2024-09-15T01:33:47.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272445+0000", 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272446+0000", 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.8299999999999995e-07 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272467+0000", 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0319999999999999e-05 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272505+0000", 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.8862999999999997e-05 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.272513+0000", 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.362e-06 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281243+0000", 2024-09-15T01:33:47.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0087304259999999995 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281250+0000", 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9399999999999996e-06 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281315+0000", 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4664000000000003e-05 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282868+0000", 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001552793 2024-09-15T01:33:47.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282917+0000", 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9567999999999997e-05 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282975+0000", 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7617000000000001e-05 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.313647+0000", 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030671894000000002 2024-09-15T01:33:47.462 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWrite(3.7s6 44/38 ECSubWrite(tid=62, reqid=client.4477.0:308, at_version=44'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:46.273497+0000", 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 61.121093463999998, 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.040229292, 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273497+0000", 2024-09-15T01:33:47.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273497+0000", 2024-09-15T01:33:47.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273498+0000", 2024-09-15T01:33:47.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.2799999999999996e-07 2024-09-15T01:33:47.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273514+0000", 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5942000000000001e-05 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273538+0000", 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.4705000000000001e-05 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.273543+0000", 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9649999999999999e-06 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281338+0000", 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0077948050000000001 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281343+0000", 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6009999999999997e-06 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.281391+0000", 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.7768000000000001e-05 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282868+0000", 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001477587 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282917+0000", 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9168000000000001e-05 2024-09-15T01:33:47.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.282987+0000", 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.9294000000000004e-05 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:46.313726+0000", 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030739829 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=100, last_complete=54'36, committed=1, applied=1))", 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.298181792999998, 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.063605834999999999, 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096409+0000", 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.01e-07 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.096412+0000", 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.836e-06 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.159953+0000", 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.063540907999999993 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.159956+0000", 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0489999999999999e-06 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.159987+0000", 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.0725999999999997e-05 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.160015+0000", 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.7915e-05 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:485 3.9s0 3:98b2f2fd:::smithi17529310-14:head [setxattr _header (57) in=7b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.230218102000002, 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40475048200000002, 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:47.472 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 485 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164373+0000", 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6199999999999998e-07 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164375+0000", 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.0650000000000001e-06 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164380+0000", 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.9810000000000003e-06 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164383+0000", 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.241e-06 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165443+0000", 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010601409999999999 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165554+0000", 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011091600000000001 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165888+0000", 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000334125 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165960+0000", 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.1601999999999996e-05 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.165971+0000", 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0903999999999999e-05 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166005+0000", 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3525999999999997e-05 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166156+0000", 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000151077 2024-09-15T01:33:47.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569047+0000", 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.402891102 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569123+0000", 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.6340000000000004e-05 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4477.0:486 3.9s0 3:98b2f2fd:::smithi17529310-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.230192051000003, 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40586654100000003, 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "started", 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4477", 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 486 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164399+0000", 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.2800000000000003e-07 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164402+0000", 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.3440000000000001e-06 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164409+0000", 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.6839999999999999e-06 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.164417+0000", 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.9140000000000008e-06 2024-09-15T01:33:47.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166021+0000", 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016044189999999999 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166112+0000", 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.0402000000000006e-05 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569421+0000", 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40330946099999998 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569534+0000", 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011237499999999999 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.570265+0000", 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00073161400000000001 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=106, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.228392974000002, 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.10267822, 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166198+0000", 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0399999999999996e-07 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166203+0000", 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.2889999999999999e-06 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268821+0000", 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.102618771 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268827+0000", 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.1140000000000002e-06 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268865+0000", 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 3.7492e-05 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.268876+0000", 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.095e-05 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=106, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.166241+0000", 2024-09-15T01:33:47.487 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.228349193, 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.075762497999999998, 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166241+0000", 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166241+0000", 2024-09-15T01:33:47.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166242+0000", 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 4.6400000000000003e-07 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166244+0000", 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.3309999999999998e-06 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.241879+0000", 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.075634921999999993 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.241898+0000", 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.9082000000000001e-05 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.241991+0000", 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.2833999999999999e-05 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.242004+0000", 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.2865e-05 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 54/31 ECSubWriteReply(tid=106, last_complete=54'41, committed=1, applied=1))", 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:32:56.166248+0000", 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.228342640999998, 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.40313786299999999, 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166248+0000", 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166248+0000", 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166249+0000", 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6700000000000003e-07 2024-09-15T01:33:47.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.166251+0000", 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.934e-06 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.568876+0000", 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.402625446 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.568887+0000", 2024-09-15T01:33:47.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0553e-05 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.568964+0000", 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.7303000000000002e-05 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:32:56.569386+0000", 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00042205999999999998 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 81/31 ECSubWriteReply(tid=148, last_complete=81'55, committed=1, applied=1))", 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2024-09-15T01:33:23.088718+0000", 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 24.305872583999999, 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.42277780999999998, 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.088718+0000", 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.088718+0000", 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.088719+0000", 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.1999999999999998e-07 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.088725+0000", 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7209999999999999e-06 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.511335+0000", 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.42261066600000002 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.511351+0000", 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.5153e-05 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.511467+0000", 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011659600000000001 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-15T01:33:47.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-15T01:33:47.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2024-09-15T01:33:47.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2024-09-15T01:33:23.511496+0000", 2024-09-15T01:33:47.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 2.8753999999999999e-05 2024-09-15T01:33:47.498 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.498 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.498 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.498 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-15T01:33:47.498 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-15T01:33:47.498 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-15T01:33:47.499 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:33:47.608 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:47.608 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:47.608 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:47.608 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:47.617 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:33:47.714 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:47.714 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:47.714 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:47.714 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:47.714 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:47.722 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:33:47.820 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:47.820 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:47.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:47.820 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:47.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.820 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:192 3.as0 3:5e9da7d3:::smithi17529310-1:head [copy-from ver 6] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-15T01:33:47.820 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.625191330000007, 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01403071, 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 192 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:47.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192421+0000", 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192423+0000", 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3140000000000002e-06 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192431+0000", 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2320000000000001e-06 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192444+0000", 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3239e-05 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192454+0000", 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2439999999999999e-06 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192781+0000", 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032732699999999998 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.192904+0000", 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012317900000000001 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.201873+0000", 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089691839999999998 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202377+0000", 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00050339000000000004 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202392+0000", 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5201999999999999e-05 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202869+0000", 2024-09-15T01:33:47.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047669800000000001 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.202888+0000", 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9202e-05 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.203431+0000", 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00054287499999999995 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:39.206451+0000", 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003020624 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 44/31 ECSubWrite(tid=37, reqid=osd.7.0:37, at_version=44'30, trim_to=0'0, roll_forward_to=40'26))", 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.570637118999997, 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013669871, 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246975+0000", 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246976+0000", 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9800000000000002e-07 2024-09-15T01:33:47.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247000+0000", 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4314000000000001e-05 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247045+0000", 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4938000000000002e-05 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247053+0000", 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:33:47.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247132+0000", 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8799000000000002e-05 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247142+0000", 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0511e-05 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247515+0000", 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037233299999999997 2024-09-15T01:33:47.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247546+0000", 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1102000000000001e-05 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247585+0000", 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9152000000000001e-05 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247603+0000", 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.260645+0000", 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013041662000000001 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=55, reqid=client.4477.0:517, at_version=55'45, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.742079611999998, 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016315308000000001, 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.832 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075532+0000", 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075533+0000", 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9199999999999999e-07 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.075557+0000", 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3264e-05 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076900+0000", 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013436749999999999 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076916+0000", 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5302999999999999e-05 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076970+0000", 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4265999999999998e-05 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076980+0000", 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0521e-05 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077104+0000", 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012322599999999999 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077118+0000", 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4742e-05 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077164+0000", 2024-09-15T01:33:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5972e-05 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.077228+0000", 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3291000000000001e-05 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.091848+0000", 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014620055999999999 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=56, reqid=client.4477.0:518, at_version=55'46, trim_to=0'0, roll_forward_to=51'42))", 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.740757977000001, 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015201957, 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076854+0000", 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076855+0000", 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.057e-06 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.076876+0000", 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0874e-05 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079870+0000", 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0029943259999999998 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079882+0000", 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2026e-05 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079958+0000", 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5263000000000001e-05 2024-09-15T01:33:47.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.079969+0000", 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.145e-05 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.080116+0000", 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014736600000000001 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.080133+0000", 2024-09-15T01:33:47.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6776e-05 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082691+0000", 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025575229999999999 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082740+0000", 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9373000000000003e-05 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.092056+0000", 2024-09-15T01:33:47.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093159230000000003 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 55/31 ECSubWrite(tid=57, reqid=client.4477.0:518, at_version=0'0, trim_to=0'0, roll_forward_to=55'46))", 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.733834250000001, 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.77065714699999999, 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083778+0000", 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083779+0000", 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-15T01:33:47.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083792+0000", 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3478e-05 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853926+0000", 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.77013421000000004 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853932+0000", 2024-09-15T01:33:47.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2820000000000004e-06 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.853992+0000", 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9886999999999998e-05 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854001+0000", 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3700000000000001e-06 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854114+0000", 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000112651 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854152+0000", 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8518e-05 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854203+0000", 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1288000000000001e-05 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854223+0000", 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9884000000000001e-05 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.854435+0000", 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021157399999999999 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s3 66/31 ECSubWrite(tid=130, reqid=osd.14.0:130, at_version=66'56, trim_to=0'0, roll_forward_to=60'52))", 2024-09-15T01:33:47.848 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 39.736817252999998, 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016749074999999999, 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080795+0000", 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8199999999999999e-07 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080811+0000", 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6435000000000001e-05 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082265+0000", 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001453415 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082270+0000", 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9470000000000002e-06 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082610+0000", 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00034021700000000003 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082618+0000", 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3969999999999994e-06 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084267+0000", 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001649069 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084292+0000", 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4125999999999999e-05 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084507+0000", 2024-09-15T01:33:47.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0002154 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084539+0000", 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2208000000000001e-05 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097544+0000", 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013004679 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 66/54 ECSubWrite(tid=240, reqid=osd.13.0:240, at_version=66'36, trim_to=0'0, roll_forward_to=56'33))", 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 39.736746537000002, 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020188490999999999, 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080866+0000", 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4600000000000001e-07 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080879+0000", 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3557000000000001e-05 2024-09-15T01:33:47.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082859+0000", 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019801799999999998 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082865+0000", 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8309999999999997e-06 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083375+0000", 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051010200000000002 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083383+0000", 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7740000000000004e-06 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085762+0000", 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023785339999999999 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086258+0000", 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00049678099999999996 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086290+0000", 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1612999999999997e-05 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086368+0000", 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8417000000000005e-05 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101054+0000", 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014685556000000001 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=62, reqid=osd.7.0:62, at_version=66'52, trim_to=0'0, roll_forward_to=58'50))", 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081340+0000", 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 39.736271719000001, 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016057799000000001, 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081340+0000", 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081340+0000", 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081341+0000", 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3900000000000004e-07 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081364+0000", 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3078000000000002e-05 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082876+0000", 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001512298 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082880+0000", 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1150000000000004e-06 2024-09-15T01:33:47.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082998+0000", 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117386 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083007+0000", 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4849999999999998e-06 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084480+0000", 2024-09-15T01:33:47.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014727799999999999 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084509+0000", 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8878e-05 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084575+0000", 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5848999999999999e-05 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085049+0000", 2024-09-15T01:33:47.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047466899999999999 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097398+0000", 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012348622 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=63, reqid=osd.7.0:63, at_version=66'54, trim_to=0'0, roll_forward_to=58'50))", 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 39.736244481, 2024-09-15T01:33:47.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017023607, 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081368+0000", 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081386+0000", 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952e-05 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083054+0000", 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001668295 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083059+0000", 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9359999999999998e-06 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084544+0000", 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001484983 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084582+0000", 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7764e-05 2024-09-15T01:33:47.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084807+0000", 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022473099999999999 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.084975+0000", 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000167997 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086254+0000", 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001279608 2024-09-15T01:33:47.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086318+0000", 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3590000000000006e-05 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.098391+0000", 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012073421000000001 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 66/31 ECSubWrite(tid=241, reqid=osd.13.0:241, at_version=66'51, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.081392+0000", 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 39.736219652999999, 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017955004, 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081392+0000", 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081392+0000", 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081393+0000", 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7399999999999999e-07 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.081416+0000", 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3807000000000001e-05 2024-09-15T01:33:47.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083070+0000", 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001653416 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083073+0000", 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.58e-06 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083228+0000", 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000154671 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083236+0000", 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1960000000000008e-06 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085387+0000", 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021502230000000002 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086257+0000", 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00087064300000000005 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086289+0000", 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2048000000000002e-05 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086344+0000", 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5031000000000002e-05 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.099347+0000", 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013003115000000001 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 66/31 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=66'53, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.083317+0000", 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 39.734294968, 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017650290999999999, 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.875 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083317+0000", 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083317+0000", 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083318+0000", 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9100000000000004e-07 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083337+0000", 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9488e-05 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083371+0000", 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3522000000000002e-05 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083382+0000", 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1715999999999999e-05 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085431+0000", 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020484309999999999 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085439+0000", 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3389999999999993e-06 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.085612+0000", 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017278700000000001 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086258+0000", 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00064593500000000004 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086290+0000", 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1813999999999997e-05 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.086355+0000", 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5405000000000001e-05 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.100967+0000", 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014612263 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.880 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=64, reqid=client.4477.0:609, at_version=66'56, trim_to=0'0, roll_forward_to=66'52))", 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.087506+0000", 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 39.730106046000003, 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014356819999999999, 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087506+0000", 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087506+0000", 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087507+0000", 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.144e-06 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087633+0000", 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012627100000000001 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087682+0000", 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8551000000000003e-05 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087690+0000", 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5979999999999999e-06 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087747+0000", 2024-09-15T01:33:47.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7376000000000001e-05 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.087758+0000", 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1065e-05 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.088106+0000", 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000347766 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.088114+0000", 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3850000000000002e-06 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091745+0000", 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003630968 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091795+0000", 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9922000000000001e-05 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101863+0000", 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010067774 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 66/54 ECSubWrite(tid=243, reqid=osd.13.0:240, at_version=0'0, trim_to=0'0, roll_forward_to=66'36))", 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 39.727960170000003, 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01224519, 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.886 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089652+0000", 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.48e-07 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089655+0000", 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089726+0000", 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0436000000000001e-05 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089731+0000", 2024-09-15T01:33:47.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.164e-06 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089770+0000", 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9110999999999999e-05 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.089777+0000", 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2599999999999999e-06 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.090039+0000", 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000261506 2024-09-15T01:33:47.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091756+0000", 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001717704 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091831+0000", 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4177000000000005e-05 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.091855+0000", 2024-09-15T01:33:47.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4300000000000001e-05 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.101897+0000", 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010042163 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.as0 76/69 ECSubWriteReply(tid=60, last_complete=76'39, committed=1, applied=1))", 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:18.085798+0000", 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.731814070999999, 2024-09-15T01:33:47.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19697366399999999, 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085798+0000", 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085798+0000", 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085799+0000", 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9199999999999999e-07 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.085804+0000", 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0440000000000003e-06 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282638+0000", 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19683383099999999 2024-09-15T01:33:47.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282645+0000", 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9800000000000001e-06 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282743+0000", 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8517999999999994e-05 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.282772+0000", 2024-09-15T01:33:47.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8399e-05 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/54 ECSubWrite(tid=304, reqid=client.4477.0:847, at_version=86'55, trim_to=0'0, roll_forward_to=85'52))", 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.134900+0000", 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.682712235, 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018173871000000001, 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.895 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134900+0000", 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134900+0000", 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134901+0000", 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0359999999999999e-06 2024-09-15T01:33:47.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.897 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.897 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134932+0000", 2024-09-15T01:33:47.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0868999999999999e-05 2024-09-15T01:33:47.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.897 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.897 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.134994+0000", 2024-09-15T01:33:47.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2161000000000003e-05 2024-09-15T01:33:47.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.897 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135000+0000", 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2319999999999998e-06 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135199+0000", 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000198438 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135207+0000", 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0830000000000004e-06 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135318+0000", 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110881 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135331+0000", 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3409e-05 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135370+0000", 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9144000000000003e-05 2024-09-15T01:33:47.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135466+0000", 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5527999999999997e-05 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.153074+0000", 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01760809 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.901 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/54 ECSubWrite(tid=305, reqid=client.4477.0:848, at_version=86'56, trim_to=0'0, roll_forward_to=85'52))", 2024-09-15T01:33:47.901 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.135237+0000", 2024-09-15T01:33:47.901 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.682374629999998, 2024-09-15T01:33:47.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018127175999999998, 2024-09-15T01:33:47.901 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.901 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.901 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.901 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.901 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135237+0000", 2024-09-15T01:33:47.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135237+0000", 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135238+0000", 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3600000000000004e-07 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135253+0000", 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5339e-05 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135283+0000", 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9445e-05 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135290+0000", 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8329999999999996e-06 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135356+0000", 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5982000000000006e-05 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135363+0000", 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7030000000000006e-06 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135446+0000", 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2695999999999996e-05 2024-09-15T01:33:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135454+0000", 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1669999999999999e-06 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135511+0000", 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7265000000000001e-05 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135519+0000", 2024-09-15T01:33:47.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6739999999999997e-06 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.153365+0000", 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017845535999999999 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/54 ECSubWrite(tid=306, reqid=client.4477.0:848, at_version=0'0, trim_to=0'0, roll_forward_to=86'56))", 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.136794+0000", 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.680817865000002, 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016821658999999999, 2024-09-15T01:33:47.906 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136794+0000", 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136794+0000", 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136795+0000", 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0890000000000001e-06 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136806+0000", 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0301e-05 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136844+0000", 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8002e-05 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136852+0000", 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3979999999999999e-06 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136958+0000", 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010578600000000001 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136969+0000", 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0767999999999999e-05 2024-09-15T01:33:47.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137111+0000", 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142922 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137120+0000", 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137317+0000", 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019695499999999999 2024-09-15T01:33:47.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137331+0000", 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.385e-05 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.153616+0000", 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016284666 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.911 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.ds1 86/31 ECSubWrite(tid=104, reqid=client.4477.0:836, at_version=0'0, trim_to=0'0, roll_forward_to=86'87))", 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.137961+0000", 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.679650841000001, 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015483966, 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137961+0000", 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137961+0000", 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137962+0000", 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4200000000000004e-07 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137975+0000", 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2993000000000001e-05 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138014+0000", 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9050000000000001e-05 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138021+0000", 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1130000000000003e-06 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138076+0000", 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4867999999999997e-05 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138087+0000", 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0623e-05 2024-09-15T01:33:47.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138238+0000", 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000150944 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.139771+0000", 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00153374 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.139849+0000", 2024-09-15T01:33:47.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7737999999999993e-05 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.139880+0000", 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0809999999999998e-05 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.153445+0000", 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013565145000000001 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.916 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 100/31 ECSubWrite(tid=342, reqid=client.4477.0:970, at_version=100'99, trim_to=0'0, roll_forward_to=100'97))", 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:42.127295+0000", 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.690316996, 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014016366000000001, 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.127295+0000", 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.127295+0000", 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.127297+0000", 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.485e-06 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.127409+0000", 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000112428 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.127449+0000", 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0151999999999998e-05 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.127457+0000", 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.157e-06 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.127503+0000", 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5442999999999998e-05 2024-09-15T01:33:47.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.127513+0000", 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0389e-05 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.127847+0000", 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033397800000000002 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.127855+0000", 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2570000000000007e-06 2024-09-15T01:33:47.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.127945+0000", 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9357999999999994e-05 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.127964+0000", 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9551999999999998e-05 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.141311+0000", 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013347167 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s6 100/31 ECSubWrite(tid=343, reqid=client.4477.0:971, at_version=100'100, trim_to=0'0, roll_forward_to=100'97))", 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:42.128178+0000", 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.6894339990000002, 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013359495000000001, 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:47.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.128178+0000", 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.128178+0000", 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.128179+0000", 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0999999999999997e-07 2024-09-15T01:33:47.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.128197+0000", 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7648000000000001e-05 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.128231+0000", 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3911000000000001e-05 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.128238+0000", 2024-09-15T01:33:47.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7149999999999998e-06 2024-09-15T01:33:47.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:47.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.128286+0000", 2024-09-15T01:33:47.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7694000000000001e-05 2024-09-15T01:33:47.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:47.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.128301+0000", 2024-09-15T01:33:47.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5529e-05 2024-09-15T01:33:47.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.128424+0000", 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012271600000000001 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.128434+0000", 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.414e-06 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.128517+0000", 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3431999999999997e-05 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.128539+0000", 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1588999999999999e-05 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:42.141538+0000", 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012998937 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:47.927 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:47.928 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:47.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:33:48.025 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:48.025 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:48.025 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:48.025 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:48.033 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:33:48.131 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:48.131 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:48.131 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:48.131 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:48.131 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:48.140 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:33:48.152 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-15T01:33:48.237 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:108 3.ds0 3:b063f44c:::smithi17529310-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 71.988570695000007, 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013756292, 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 108 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246458+0000", 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246459+0000", 2024-09-15T01:33:48.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6800000000000001e-07 2024-09-15T01:33:48.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246461+0000", 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.131e-06 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246468+0000", 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5529999999999998e-06 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.246475+0000", 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5830000000000001e-06 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.248249+0000", 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017739990000000001 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.248333+0000", 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4288000000000003e-05 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.249154+0000", 2024-09-15T01:33:48.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00082014900000000005 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.249218+0000", 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4519999999999999e-05 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.260215+0000", 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010996601 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=57, reqid=osd.4.0:57, at_version=44'17, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.993806192000001, 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014051065, 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241223+0000", 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241243+0000", 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0126e-05 2024-09-15T01:33:48.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241422+0000", 2024-09-15T01:33:48.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017906899999999999 2024-09-15T01:33:48.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241429+0000", 2024-09-15T01:33:48.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4139999999999999e-06 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241500+0000", 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1098000000000003e-05 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241510+0000", 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8470000000000008e-06 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.241787+0000", 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027729700000000003 2024-09-15T01:33:48.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242007+0000", 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000220134 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242146+0000", 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013846799999999999 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.242278+0000", 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0001326 2024-09-15T01:33:48.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.255274+0000", 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01299575 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=44'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.991347847, 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014188612, 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.249 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243681+0000", 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243682+0000", 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0299999999999999e-07 2024-09-15T01:33:48.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.243699+0000", 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7575999999999999e-05 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244099+0000", 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039909500000000003 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244108+0000", 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3209999999999994e-06 2024-09-15T01:33:48.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244168+0000", 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9688999999999997e-05 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.244180+0000", 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2031000000000001e-05 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245504+0000", 2024-09-15T01:33:48.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013241520000000001 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245537+0000", 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3287000000000002e-05 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245645+0000", 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108435 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.245664+0000", 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8329999999999999e-05 2024-09-15T01:33:48.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.257870+0000", 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012206092999999999 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.bs5 44/38 ECSubWrite(tid=59, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=44'19))", 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.988064264000002, 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013797696999999999, 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.254 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246965+0000", 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246966+0000", 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000002e-07 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.246979+0000", 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3111e-05 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247017+0000", 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8096999999999998e-05 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247025+0000", 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9100000000000005e-06 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247095+0000", 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0049000000000006e-05 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247104+0000", 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7919999999999998e-06 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247217+0000", 2024-09-15T01:33:48.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011299 2024-09-15T01:33:48.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247376+0000", 2024-09-15T01:33:48.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015968800000000001 2024-09-15T01:33:48.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.247417+0000", 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0228999999999997e-05 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.248151+0000", 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00073475999999999999 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.260763+0000", 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012611282999999999 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:283 3.ds0 3:bfa066a3:::smithi17529310-39:head [copy-from ver 2] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.976263525999997, 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015911307999999999, 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 283 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258766+0000", 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258768+0000", 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0980000000000001e-06 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258779+0000", 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1272e-05 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258795+0000", 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5679999999999999e-05 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258802+0000", 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9029999999999998e-06 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.258879+0000", 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7035999999999999e-05 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.259139+0000", 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260456 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268247+0000", 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0091076109999999998 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268518+0000", 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027146499999999998 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268542+0000", 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3601e-05 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.268616+0000", 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3944999999999998e-05 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.269087+0000", 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047095399999999999 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.274577+0000", 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0054903019999999999 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.274677+0000", 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9984999999999999e-05 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 44/31 ECSubWriteReply(tid=43, last_complete=44'32, committed=1, applied=1))", 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.959582660000002, 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.192594981, 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275446+0000", 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275447+0000", 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2099999999999995e-07 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.275452+0000", 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2050000000000004e-06 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.467889+0000", 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19243722999999999 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.467922+0000", 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3237e-05 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.468017+0000", 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4543999999999995e-05 2024-09-15T01:33:48.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.468041+0000", 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4844000000000001e-05 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.3s1 66/31 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=66'53, trim_to=0'0, roll_forward_to=58'49))", 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 40.152347435000003, 2024-09-15T01:33:48.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014557652000000001, 2024-09-15T01:33:48.270 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.270 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.270 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:48.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082682+0000", 2024-09-15T01:33:48.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082683+0000", 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2999999999999999e-07 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082704+0000", 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1381000000000001e-05 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082736+0000", 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2239000000000001e-05 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082743+0000", 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6449999999999999e-06 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082927+0000", 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018380800000000001 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.082934+0000", 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1600000000000001e-06 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083133+0000", 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019963000000000001 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083536+0000", 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040219000000000002 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083587+0000", 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1600000000000001e-05 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.083644+0000", 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6753999999999999e-05 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.097239+0000", 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013595415 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.6s5 71/69 ECSubWrite(tid=73, reqid=client.4477.0:701, at_version=71'45, trim_to=0'0, roll_forward_to=70'43))", 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.321241+0000", 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 34.913787698999997, 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014395221999999999, 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321241+0000", 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321241+0000", 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321242+0000", 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.36e-07 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327486+0000", 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0062439130000000002 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327562+0000", 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6878999999999994e-05 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327570+0000", 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3520000000000001e-06 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327617+0000", 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6839999999999999e-05 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.327627+0000", 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.927e-06 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328144+0000", 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051762900000000003 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328237+0000", 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3153999999999996e-05 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328549+0000", 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000311873 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328572+0000", 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2751999999999998e-05 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.335637+0000", 2024-09-15T01:33:48.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0070647669999999996 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:823 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [copy-from ver 45] snapc 30=[30,2e,2c,2b,29,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:27.126788+0000", 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.108241397, 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.02620683, 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.280 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 823 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126788+0000", 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126788+0000", 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126790+0000", 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9360000000000002e-06 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126797+0000", 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0330000000000003e-06 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126824+0000", 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7637e-05 2024-09-15T01:33:48.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126834+0000", 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.268e-06 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126932+0000", 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7882999999999998e-05 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127109+0000", 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017735299999999999 2024-09-15T01:33:48.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.142551+0000", 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015441939 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143350+0000", 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00079953299999999995 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143360+0000", 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0098999999999999e-05 2024-09-15T01:33:48.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143663+0000", 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030244499999999998 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143835+0000", 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017216200000000001 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:48.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.152928+0000", 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0090929219999999998 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.152995+0000", 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6619999999999996e-05 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:824 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:48.286 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.108183343, 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016980479, 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 824 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126846+0000", 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6199999999999998e-07 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.126849+0000", 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6800000000000002e-06 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127309+0000", 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045968300000000001 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127316+0000", 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9500000000000004e-06 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127358+0000", 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1990999999999998e-05 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.127413+0000", 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5448999999999997e-05 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143574+0000", 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016161045999999998 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143678+0000", 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103719 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:27.143826+0000", 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014839900000000001 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:834 3.ds0 3:bfa066a3:::smithi17529310-39:head [append 0~475136] snapc 32=[32,30,2e,2c,2b,29,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.122940+0000", 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.112089210000001, 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01465559, 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.291 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 834 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.122940+0000", 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.122940+0000", 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.122941+0000", 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.187e-06 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123093+0000", 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000152087 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123103+0000", 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4339999999999998e-06 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123108+0000", 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123144+0000", 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5818000000000002e-05 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123231+0000", 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7831000000000007e-05 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123712+0000", 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00048043899999999999 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123895+0000", 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018332600000000001 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.123903+0000", 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8849999999999999e-06 2024-09-15T01:33:48.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.124026+0000", 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012307799999999999 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.124343+0000", 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031695200000000001 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:48.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137541+0000", 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013197690999999999 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.137596+0000", 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4832e-05 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:837 3.ds0 3:bfa066a3:::smithi17529310-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.125675+0000", 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.109354493000001, 2024-09-15T01:33:48.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016649648999999999, 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 837 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125675+0000", 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125675+0000", 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125675+0000", 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6499999999999999e-07 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125678+0000", 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0800000000000002e-06 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125688+0000", 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9580000000000002e-06 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125694+0000", 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4670000000000003e-06 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.127414+0000", 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017203640000000001 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.127494+0000", 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0179999999999995e-05 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138114+0000", 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010619161 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.138176+0000", 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2644999999999994e-05 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.142324+0000", 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0041481290000000004 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:842 3.ds0 3:b063f44c:::smithi17529310-36:head [read 0~2949120,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.125725+0000", 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.109304547000001, 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013984874, 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.302 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 842 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125725+0000", 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125725+0000", 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125725+0000", 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1800000000000001e-07 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125728+0000", 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125734+0000", 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6789999999999999e-06 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125738+0000", 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.126e-06 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.128322+0000", 2024-09-15T01:33:48.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025834970000000001 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.128409+0000", 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7362999999999997e-05 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.139709+0000", 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01130014 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:843 3.ds0 3:b063f44c:::smithi17529310-36:head [read 0~2949120,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.125742+0000", 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.109287041000002, 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015933147000000002, 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 843 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125742+0000", 2024-09-15T01:33:48.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125742+0000", 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125742+0000", 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9900000000000002e-07 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125745+0000", 2024-09-15T01:33:48.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2900000000000001e-06 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125751+0000", 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0310000000000004e-06 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.125755+0000", 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8190000000000002e-06 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.129247+0000", 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0034922130000000001 2024-09-15T01:33:48.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.129335+0000", 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8098999999999998e-05 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.141675+0000", 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012340396 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.310 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 90/31 ECSubWriteReply(tid=117, last_complete=90'103, committed=1, applied=1))", 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:32.087896+0000", 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 16.147133501999999, 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.78023241700000001, 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.087896+0000", 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.087896+0000", 2024-09-15T01:33:48.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.087896+0000", 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8899999999999999e-07 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.087900+0000", 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0180000000000003e-06 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.867952+0000", 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.78005183600000005 2024-09-15T01:33:48.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.867963+0000", 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1214e-05 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.868082+0000", 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118872 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:32.868128+0000", 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5887999999999998e-05 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.313 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:888 3.ds0 3:b50d79c0:::smithi17529310-18:head [read 0~2228224,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:33.082507+0000", 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.152522378, 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024728772999999999, 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 888 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.314 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082507+0000", 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082507+0000", 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082507+0000", 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.46e-07 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082509+0000", 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1339999999999998e-06 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082514+0000", 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7779999999999998e-06 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082518+0000", 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2289999999999997e-06 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082626+0000", 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107278 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.082630+0000", 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3000000000000003e-06 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.085853+0000", 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0032233330000000001 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.085981+0000", 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127572 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.107236+0000", 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021254802999999999 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:898 3.ds0 3:b063f44c:::smithi17529310-36:19 [read 0~589824,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:33.083177+0000", 2024-09-15T01:33:48.318 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.151852113, 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024976851000000001, 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 898 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083177+0000", 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083177+0000", 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083178+0000", 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.006e-06 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083184+0000", 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9150000000000001e-06 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083199+0000", 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4789e-05 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083205+0000", 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2330000000000003e-06 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083312+0000", 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010702899999999999 2024-09-15T01:33:48.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083319+0000", 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0319999999999998e-06 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.086360+0000", 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030408840000000002 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.086443+0000", 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2886999999999994e-05 2024-09-15T01:33:48.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.108154+0000", 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021711075999999999 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:899 3.ds0 3:b063f44c:::smithi17529310-36:19 [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:33.083217+0000", 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.151811840000001, 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.025600806, 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.323 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 899 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083217+0000", 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083217+0000", 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083218+0000", 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5199999999999997e-07 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083221+0000", 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9249999999999999e-06 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083228+0000", 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5340000000000002e-06 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083234+0000", 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1970000000000004e-06 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083361+0000", 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127136 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:48.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083366+0000", 2024-09-15T01:33:48.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8949999999999997e-06 2024-09-15T01:33:48.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.093115+0000", 2024-09-15T01:33:48.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0097499279999999997 2024-09-15T01:33:48.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.093210+0000", 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4549000000000006e-05 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.108818+0000", 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01560809 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:900 3.ds0 3:b063f44c:::smithi17529310-36:19 [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:33.083238+0000", 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.151790849999999, 2024-09-15T01:33:48.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.026108421, 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 900 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083238+0000", 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083238+0000", 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083239+0000", 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5200000000000002e-07 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083242+0000", 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0280000000000001e-06 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083248+0000", 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6900000000000003e-06 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083253+0000", 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1280000000000001e-06 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083395+0000", 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014244099999999999 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:48.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.083403+0000", 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8169999999999999e-06 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.093539+0000", 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010135933999999999 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.093696+0000", 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000157427 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:33.109347+0000", 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015650503999999999 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:916 3.ds0 3:bfd0d5a8:::smithi17529310-21:head [copy-from ver 60] snapc 39=[39,34,32,2e,2c,29,1f,19] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:35.104005+0000", 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.131024537, 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013890008000000001, 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.333 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 916 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104005+0000", 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104005+0000", 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104007+0000", 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1859999999999999e-06 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104018+0000", 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1613e-05 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104040+0000", 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1548000000000001e-05 2024-09-15T01:33:48.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104049+0000", 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9209999999999998e-06 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104099+0000", 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0553000000000002e-05 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.104180+0000", 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0117999999999994e-05 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.111941+0000", 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0077609970000000004 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.112308+0000", 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036721600000000002 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.112317+0000", 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8650000000000007e-06 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.112391+0000", 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4833999999999995e-05 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.112420+0000", 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8977999999999999e-05 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.114820+0000", 2024-09-15T01:33:48.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023993360000000002 2024-09-15T01:33:48.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:35.117895+0000", 2024-09-15T01:33:48.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030748429999999998 2024-09-15T01:33:48.339 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.339 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.339 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.339 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.339 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.339 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:48.340 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a: {} 2024-09-15T01:33:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T01:33:48.341 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b: {} 2024-09-15T01:33:48.341 INFO:teuthology.orchestra.run.smithi042.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T01:33:48.342 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:33:48.342 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c: {} 2024-09-15T01:33:48.343 INFO:teuthology.orchestra.run.smithi042.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T01:33:48.343 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T01:33:48.441 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:48.441 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:48.441 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:48.441 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:48.448 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:33:48.546 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:48.546 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:48.546 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:48.546 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:48.546 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:48.554 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:33:48.651 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:48.651 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:48.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:48.651 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:48.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.651 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:99 3.fs0 3:f630fe6e:::smithi17529310-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:48.651 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:48.651 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 72.413177715000003, 2024-09-15T01:33:48.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055995037999999997, 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235202+0000", 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235203+0000", 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4000000000000002e-07 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235205+0000", 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3e-06 2024-09-15T01:33:48.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235212+0000", 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4389999999999997e-06 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.235216+0000", 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.031e-06 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.236877+0000", 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00166088 2024-09-15T01:33:48.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:48.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.236933+0000", 2024-09-15T01:33:48.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6873999999999998e-05 2024-09-15T01:33:48.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255906+0000", 2024-09-15T01:33:48.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018972066999999999 2024-09-15T01:33:48.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255957+0000", 2024-09-15T01:33:48.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1467e-05 2024-09-15T01:33:48.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.291198+0000", 2024-09-15T01:33:48.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035240540000000001 2024-09-15T01:33:48.656 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.656 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.656 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:114 3.fs0 3:f8072b57:::smithi17529310-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 72.394737793999994, 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048699039999999999, 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 114 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.657 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253642+0000", 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253643+0000", 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6899999999999998e-07 2024-09-15T01:33:48.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253646+0000", 2024-09-15T01:33:48.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-15T01:33:48.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253652+0000", 2024-09-15T01:33:48.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3369999999999996e-06 2024-09-15T01:33:48.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.253656+0000", 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.031e-06 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255410+0000", 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017543420000000001 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.255505+0000", 2024-09-15T01:33:48.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4704999999999995e-05 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290833+0000", 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035327711999999997 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.290893+0000", 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0384999999999999e-05 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.302341+0000", 2024-09-15T01:33:48.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011448274 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:145 3.cs0 3:34a45b5f:::smithi17529310-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 72.350511225999995, 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15351202899999999, 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.662 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 145 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297869+0000", 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0500000000000004e-07 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297871+0000", 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9769999999999999e-06 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297878+0000", 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9120000000000001e-06 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.297882+0000", 2024-09-15T01:33:48.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0910000000000003e-06 2024-09-15T01:33:48.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.298932+0000", 2024-09-15T01:33:48.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00104904 2024-09-15T01:33:48.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:48.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.299009+0000", 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7095000000000006e-05 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.306936+0000", 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0079278730000000002 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.307022+0000", 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5254999999999996e-05 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.451381+0000", 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14435928100000001 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:312 3.cs0 3:37dffec7:::smithi17529310-13:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 62.370187610999999, 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770305000000002, 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.667 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 312 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278193+0000", 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0200000000000002e-07 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278196+0000", 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7319999999999998e-06 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278216+0000", 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0375999999999999e-05 2024-09-15T01:33:48.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278221+0000", 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4889999999999997e-06 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279668+0000", 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014473929999999999 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.279753+0000", 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4567999999999994e-05 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280213+0000", 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00046030399999999998 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280327+0000", 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011404399999999999 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280342+0000", 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4403000000000001e-05 2024-09-15T01:33:48.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280467+0000", 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000125594 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280562+0000", 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5317999999999998e-05 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328870+0000", 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048307782 2024-09-15T01:33:48.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328963+0000", 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2700000000000004e-05 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:313 3.cs0 3:37dffec7:::smithi17529310-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 62.370154730000003, 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053267040000000002, 2024-09-15T01:33:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 313 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278225+0000", 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278226+0000", 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7100000000000002e-07 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278228+0000", 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1859999999999999e-06 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278235+0000", 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9020000000000002e-06 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.278240+0000", 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9540000000000003e-06 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280404+0000", 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021641469999999999 2024-09-15T01:33:48.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.280503+0000", 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9090000000000002e-05 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328996+0000", 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048492533999999997 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.329053+0000", 2024-09-15T01:33:48.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6997999999999999e-05 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.331493+0000", 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024397580000000002 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=95, last_complete=44'34, committed=1, applied=1))", 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:48.678 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 62.366750387000003, 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047344538999999998, 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281630+0000", 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7700000000000001e-07 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.281632+0000", 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.562e-06 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328775+0000", 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047143813 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328779+0000", 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.895e-06 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328803+0000", 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4076000000000001e-05 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.328974+0000", 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017101599999999999 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:324 3.cs0 3:34a45b5f:::smithi17529310-48:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 62.317735280000001, 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18862163400000001, 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.682 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 324 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330645+0000", 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699999999999999e-07 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330648+0000", 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.694e-06 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330654+0000", 2024-09-15T01:33:48.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4090000000000002e-06 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330659+0000", 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3210000000000004e-06 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332198+0000", 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015388699999999999 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332262+0000", 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4196000000000007e-05 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332785+0000", 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052292099999999998 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332870+0000", 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5049999999999994e-05 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332877+0000", 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7769999999999997e-06 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332902+0000", 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5012e-05 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.332962+0000", 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0649999999999997e-05 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519220+0000", 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18625739199999999 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519267+0000", 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6875000000000001e-05 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:325 3.cs0 3:34a45b5f:::smithi17529310-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e44)", 2024-09-15T01:33:48.688 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 62.317717217000002, 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.189890015, 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 325 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330663+0000", 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330667+0000", 2024-09-15T01:33:48.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4410000000000002e-06 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330673+0000", 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5429999999999999e-06 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.330676+0000", 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.683e-06 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333439+0000", 2024-09-15T01:33:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002763415 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333492+0000", 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2760000000000003e-05 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519496+0000", 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18600408199999999 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519560+0000", 2024-09-15T01:33:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3660999999999999e-05 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.520553+0000", 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00099301399999999996 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 44/31 ECSubWriteReply(tid=100, last_complete=44'39, committed=1, applied=1))", 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 62.315315173999998, 2024-09-15T01:33:48.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18640773799999999, 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333065+0000", 2024-09-15T01:33:48.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5999999999999999e-08 2024-09-15T01:33:48.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.333066+0000", 2024-09-15T01:33:48.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.043e-06 2024-09-15T01:33:48.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519088+0000", 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18602205399999999 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519092+0000", 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0890000000000002e-06 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519171+0000", 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8857999999999996e-05 2024-09-15T01:33:48.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:46.519473+0000", 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030159799999999998 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 47/31 ECSubWriteReply(tid=113, last_complete=47'51, committed=1, applied=1))", 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 59.388493363000002, 2024-09-15T01:33:48.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.63648575600000001, 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259887+0000", 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-07 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.259890+0000", 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5859999999999999e-06 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.895924+0000", 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.63603432699999995 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.895929+0000", 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1340000000000003e-06 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.896000+0000", 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0687999999999998e-05 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:49.896373+0000", 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037265200000000001 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 48/31 ECSubWriteReply(tid=128, last_complete=48'57, committed=1, applied=1))", 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 58.341677873999998, 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18111660500000001, 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.701 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306702+0000", 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306703+0000", 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3e-07 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.306707+0000", 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7500000000000001e-06 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487741+0000", 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18103418199999999 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487745+0000", 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2189999999999998e-06 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487789+0000", 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3430000000000003e-05 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:50.487819+0000", 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0394000000000001e-05 2024-09-15T01:33:48.704 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 53/31 ECSubWriteReply(tid=146, last_complete=53'74, committed=1, applied=1))", 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 53.447836424999998, 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.74248160299999999, 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200544+0000", 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-07 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.200550+0000", 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5910000000000003e-06 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.942942+0000", 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.74239190499999996 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.942965+0000", 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2983999999999999e-05 2024-09-15T01:33:48.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.943000+0000", 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5336999999999997e-05 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:55.943025+0000", 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5250999999999999e-05 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.7s6 55/52 [PushOp(3:e6e8875f:::smithi17529310-31:head, version: 50'31, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:e6e8875f:::smithi17529310-31:head@50'31, size: 1949696, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.550121820000001, 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18071695199999999, 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098258+0000", 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098259+0000", 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0200000000000001e-07 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.098545+0000", 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028574699999999999 2024-09-15T01:33:48.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278693+0000", 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.180148009 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278700+0000", 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.888e-06 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278803+0000", 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102766 2024-09-15T01:33:48.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.278975+0000", 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017284000000000001 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:524 3.fs0 3:f8e751c8:::smithi17529310-17:head [setxattr _header (57) in=7b] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.554357347, 2024-09-15T01:33:48.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.115556992, 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 524 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094023+0000", 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2200000000000004e-07 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094028+0000", 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9710000000000003e-06 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094040+0000", 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1066e-05 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094044+0000", 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8770000000000001e-06 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095122+0000", 2024-09-15T01:33:48.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010771839999999999 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095188+0000", 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6298999999999997e-05 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095521+0000", 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033339600000000002 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095614+0000", 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2295999999999999e-05 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095620+0000", 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8000000000000001e-06 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095653+0000", 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2867000000000002e-05 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095740+0000", 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6300999999999999e-05 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209497+0000", 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.113757184 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209580+0000", 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3128999999999996e-05 2024-09-15T01:33:48.718 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:525 3.fs0 3:f8e751c8:::smithi17529310-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.554331521999998, 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11664384, 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.719 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 525 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094049+0000", 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6100000000000001e-07 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094051+0000", 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1950000000000002e-06 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094058+0000", 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4350000000000002e-06 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.094063+0000", 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2039999999999996e-06 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095664+0000", 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001600642 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.095725+0000", 2024-09-15T01:33:48.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1707999999999999e-05 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209923+0000", 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.114197744 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.209997+0000", 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3341000000000001e-05 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.210693+0000", 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00069601000000000005 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:691 3.fs0 3:f73c1a72:::smithi17529310-29:head [copy-from ver 48] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.296702+0000", 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.351677944000002, 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.042728760999999997, 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.724 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 691 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296702+0000", 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296702+0000", 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296703+0000", 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296704+0000", 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7039999999999999e-06 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296708+0000", 2024-09-15T01:33:48.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0439999999999997e-06 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.296710+0000", 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0200000000000001e-06 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.299482+0000", 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027719820000000001 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.299578+0000", 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5361999999999999e-05 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.321672+0000", 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022094721000000001 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322236+0000", 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00056322499999999999 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322250+0000", 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.438e-05 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322630+0000", 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00038026599999999998 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.322764+0000", 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013368899999999999 2024-09-15T01:33:48.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.339358+0000", 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016593988 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.339431+0000", 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3041999999999996e-05 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.730 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:746 3.9s0 3:98b2f2fd:::smithi17529310-14:head [append 0~0] snapc 27=[27,26,24,22,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:18.078177+0000", 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 30.570203067000001, 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0091383140000001, 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 746 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078177+0000", 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078177+0000", 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078178+0000", 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1799999999999995e-07 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078181+0000", 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.038e-06 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078187+0000", 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4760000000000003e-06 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078191+0000", 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7189999999999999e-06 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078448+0000", 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025739300000000002 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op must wait for map", 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078457+0000", 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7590000000000003e-06 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.082114+0000", 2024-09-15T01:33:48.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036565579999999999 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.082117+0000", 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0299999999999998e-06 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.084154+0000", 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0020370750000001 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.084312+0000", 2024-09-15T01:33:48.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015769300000000001 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.084835+0000", 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052329899999999996 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.085132+0000", 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029761499999999999 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.085143+0000", 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0951000000000001e-05 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.085193+0000", 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9369000000000001e-05 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.086029+0000", 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00083588200000000003 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:48.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.086781+0000", 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00075188200000000005 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087315+0000", 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00053495699999999997 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:747 3.9s0 3:98b2f2fd:::smithi17529310-14:head [rollback 19] snapc 27=[27,26,24,22,1f,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:48.738 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 30.569906672999998, 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.011011909, 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 747 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:48.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078474+0000", 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8299999999999995e-07 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078479+0000", 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5549999999999996e-06 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078489+0000", 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0339999999999999e-05 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078495+0000", 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4999999999999999e-06 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078537+0000", 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2194999999999998e-05 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.078544+0000", 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0759999999999997e-06 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.084273+0000", 2024-09-15T01:33:48.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057284170000000004 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:18.084275+0000", 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4049999999999998e-06 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087195+0000", 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.002919946 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:48.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087315+0000", 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000119883 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.087944+0000", 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00062966699999999999 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088093+0000", 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000148531 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088104+0000", 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1381e-05 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088151+0000", 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6495000000000001e-05 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.088666+0000", 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051494400000000004 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.089420+0000", 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00075379099999999997 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:19.089485+0000", 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5900000000000003e-05 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 81/76 ECSubWriteReply(tid=313, last_complete=81'63, committed=1, applied=1))", 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:23.083561+0000", 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 25.564818800000001, 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87779885599999996, 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083561+0000", 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083561+0000", 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083562+0000", 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5400000000000002e-07 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.083564+0000", 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4739999999999999e-06 2024-09-15T01:33:48.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.960927+0000", 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87736222600000002 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.960933+0000", 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3550000000000001e-06 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.961015+0000", 2024-09-15T01:33:48.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1646999999999999e-05 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:23.961360+0000", 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0003457 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 87/31 ECSubWriteReply(tid=365, last_complete=87'192, committed=1, applied=1))", 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:29.144736+0000", 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.503644616999999, 2024-09-15T01:33:48.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.52162740600000002, 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.144736+0000", 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.144736+0000", 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.144736+0000", 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2900000000000004e-07 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.144740+0000", 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4300000000000002e-06 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.665922+0000", 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.52118211400000003 2024-09-15T01:33:48.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.665927+0000", 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3469999999999998e-06 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.666003+0000", 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6185000000000003e-05 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:29.666363+0000", 2024-09-15T01:33:48.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035980099999999998 2024-09-15T01:33:48.754 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.754 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.754 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.754 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:48.754 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:48.754 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:48.755 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-15T01:33:48.851 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-15T01:33:48.852 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:48.860 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-15T01:33:48.956 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:48.956 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-15T01:33:48.956 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-15T01:33:48.956 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-15T01:33:48.956 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:48.964 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-15T01:33:49.062 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-15T01:33:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-15T01:33:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-15T01:33:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-15T01:33:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:11 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 72.882569200000006, 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032033109999999997, 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 11 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176989+0000", 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176990+0000", 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2200000000000002e-07 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.176995+0000", 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8040000000000001e-06 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177006+0000", 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1276e-05 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177012+0000", 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9370000000000001e-06 2024-09-15T01:33:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177055+0000", 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2472999999999998e-05 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177197+0000", 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142519 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177567+0000", 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036945000000000001 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177659+0000", 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1928999999999993e-05 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177694+0000", 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4885000000000002e-05 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177722+0000", 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8334e-05 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177733+0000", 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1491000000000001e-05 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208963+0000", 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031230106000000001 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209022+0000", 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8984000000000003e-05 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:12 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 72.881818932000002, 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033440417, 2024-09-15T01:33:49.069 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 12 2024-09-15T01:33:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177740+0000", 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2500000000000005e-07 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177745+0000", 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7269999999999998e-06 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177755+0000", 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9000000000000001e-06 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177760+0000", 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1150000000000002e-06 2024-09-15T01:33:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177801+0000", 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0802e-05 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.177877+0000", 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6604e-05 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209067+0000", 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031189505999999999 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.209144+0000", 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6728000000000001e-05 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211180+0000", 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020363099999999999 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=12, reqid=client.4477.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 72.869614718999998, 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022201101000000001, 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189944+0000", 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.75e-07 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.189957+0000", 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2531000000000001e-05 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211941+0000", 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021983546999999999 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211947+0000", 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7329999999999997e-06 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211977+0000", 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9607e-05 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.211982+0000", 2024-09-15T01:33:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6290000000000001e-06 2024-09-15T01:33:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212053+0000", 2024-09-15T01:33:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1890000000000005e-05 2024-09-15T01:33:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212060+0000", 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3339999999999998e-06 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212087+0000", 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7314e-05 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212098+0000", 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1141999999999999e-05 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212145+0000", 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699000000000001e-05 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:72 3.5s0 3:af8bec4d:::smithi17529310-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 72.852241985000006, 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.037268951000000002, 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207317+0000", 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.34e-07 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207319+0000", 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207326+0000", 2024-09-15T01:33:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4860000000000002e-06 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.207330+0000", 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0690000000000003e-06 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208687+0000", 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001356966 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.208760+0000", 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2663000000000005e-05 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210380+0000", 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001620659 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.210464+0000", 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3387999999999995e-05 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.244586+0000", 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034121925999999997 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.0s2 34/31 ECSubWrite(tid=15, reqid=client.4477.0:74, at_version=34'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 72.847131238000003, 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069361051000000007, 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212427+0000", 2024-09-15T01:33:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212428+0000", 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3399999999999999e-07 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.212444+0000", 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5681000000000001e-05 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281357+0000", 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068913881999999996 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281366+0000", 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9740000000000008e-06 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281415+0000", 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8498000000000002e-05 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281421+0000", 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4590000000000003e-06 2024-09-15T01:33:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281487+0000", 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5590000000000001e-05 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281500+0000", 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3298e-05 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281575+0000", 2024-09-15T01:33:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5247000000000006e-05 2024-09-15T01:33:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281597+0000", 2024-09-15T01:33:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1872999999999999e-05 2024-09-15T01:33:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:36.281788+0000", 2024-09-15T01:33:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000191015 2024-09-15T01:33:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 38/31 ECSubWrite(tid=17, reqid=client.4477.0:208, at_version=38'12, trim_to=0'0, roll_forward_to=36'8))", 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.838197339999994, 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.93429882900000005, 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221361+0000", 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221362+0000", 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9699999999999996e-07 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:40.221380+0000", 2024-09-15T01:33:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.804e-05 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.154929+0000", 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.93354887600000003 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.154938+0000", 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6220000000000004e-06 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155018+0000", 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0136000000000007e-05 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155027+0000", 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8259999999999999e-06 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155116+0000", 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8939999999999999e-05 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155127+0000", 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0981e-05 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155185+0000", 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7828999999999998e-05 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155204+0000", 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8649999999999999e-05 2024-09-15T01:33:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:41.155660+0000", 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000456432 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.8s5 51/31 ECSubWrite(tid=56, reqid=osd.5.0:56, at_version=51'43, trim_to=0'0, roll_forward_to=50'41))", 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.982707605999998, 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.063539848999999995, 2024-09-15T01:33:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076851+0000", 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.076876+0000", 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4834999999999999e-05 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139053+0000", 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.062177018000000001 2024-09-15T01:33:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139067+0000", 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3842999999999999e-05 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139206+0000", 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013896500000000001 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139221+0000", 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5193e-05 2024-09-15T01:33:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139472+0000", 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025116 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139504+0000", 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1282999999999998e-05 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139541+0000", 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7518999999999998e-05 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.139557+0000", 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5781e-05 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:53.140391+0000", 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00083395699999999997 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.101 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:505 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [copy-from ver 2] snapc 15=[15,14] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.986520263000003, 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15814687399999999, 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 505 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.102 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073038+0000", 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073040+0000", 2024-09-15T01:33:49.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.792e-06 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073047+0000", 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0169999999999999e-06 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073064+0000", 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.668e-05 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073081+0000", 2024-09-15T01:33:49.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7000999999999999e-05 2024-09-15T01:33:49.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073158+0000", 2024-09-15T01:33:49.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6779000000000005e-05 2024-09-15T01:33:49.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:49.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.073284+0000", 2024-09-15T01:33:49.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000126802 2024-09-15T01:33:49.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:49.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082517+0000", 2024-09-15T01:33:49.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0092324069999999998 2024-09-15T01:33:49.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082822+0000", 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030544099999999999 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082921+0000", 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8760999999999999e-05 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.082994+0000", 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2769999999999996e-05 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.083217+0000", 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000223706 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231110+0000", 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.147892096 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231185+0000", 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5622000000000002e-05 2024-09-15T01:33:49.108 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 55/31 ECSubWriteReply(tid=49, last_complete=55'33, committed=1, applied=1))", 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.975276950000001, 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14718672399999999, 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084282+0000", 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9599999999999998e-07 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.084287+0000", 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.230928+0000", 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14664121799999999 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.230933+0000", 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4089999999999996e-06 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231018+0000", 2024-09-15T01:33:49.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5589999999999999e-05 2024-09-15T01:33:49.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:57.231468+0000", 2024-09-15T01:33:49.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045001999999999998 2024-09-15T01:33:49.112 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.112 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.112 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s4 56/54 ECSubWrite(tid=228, reqid=client.4477.0:520, at_version=56'33, trim_to=0'0, roll_forward_to=55'31))", 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.985556918999997, 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016830581000000001, 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074002+0000", 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7000000000000003e-07 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074027+0000", 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4845e-05 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074072+0000", 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4583000000000003e-05 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074080+0000", 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6699999999999994e-06 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074280+0000", 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020015699999999999 2024-09-15T01:33:49.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074290+0000", 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0586e-05 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074562+0000", 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027202599999999997 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:49.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074573+0000", 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0974e-05 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074624+0000", 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0385e-05 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.074786+0000", 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016278399999999999 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:32:58.090832+0000", 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016045700999999999 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.9s4 66/31 ECSubWrite(tid=130, reqid=osd.14.0:130, at_version=66'56, trim_to=0'0, roll_forward_to=60'52))", 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 40.979299259000001, 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.31166652299999997, 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080259+0000", 2024-09-15T01:33:49.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2000000000000001e-07 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.080275+0000", 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.556e-05 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.390960+0000", 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.31068479999999998 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.390970+0000", 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0223e-05 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391016+0000", 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5972e-05 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391025+0000", 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8610000000000004e-06 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391232+0000", 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020709799999999999 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391264+0000", 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2347e-05 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391319+0000", 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5096999999999998e-05 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391336+0000", 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6212e-05 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:08.391926+0000", 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00059013299999999996 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.123 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:646 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [copy-from ver 59] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.077480+0000", 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.982078964999999, 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024635151000000001, 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 646 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077480+0000", 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077480+0000", 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077482+0000", 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.967e-06 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077494+0000", 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2351000000000001e-05 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077510+0000", 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6461e-05 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077528+0000", 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7297000000000001e-05 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077583+0000", 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4877999999999999e-05 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op must wait for map", 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077588+0000", 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5809999999999996e-06 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.078844+0000", 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001255693 2024-09-15T01:33:49.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.078981+0000", 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013696499999999999 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097046+0000", 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018065193 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097686+0000", 2024-09-15T01:33:49.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00063973600000000002 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097704+0000", 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7934000000000001e-05 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098011+0000", 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030703799999999998 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:49.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098161+0000", 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000150233 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.102063+0000", 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003902109 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.102115+0000", 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1715000000000002e-05 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.130 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:647 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.077541+0000", 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.982017818000003, 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020616064, 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 647 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077541+0000", 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077541+0000", 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077542+0000", 2024-09-15T01:33:49.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2099999999999995e-07 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077545+0000", 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5360000000000001e-06 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077555+0000", 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0193e-05 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077561+0000", 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5679999999999999e-06 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077630+0000", 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9648999999999997e-05 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.077635+0000", 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0050000000000004e-06 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.079228+0000", 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015930009999999999 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for blocked object", 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.079271+0000", 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2588e-05 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.097903+0000", 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018632124999999999 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098048+0000", 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000144979 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.098157+0000", 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010859899999999999 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:676 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [setxattr _header (56) in=7b] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:49.136 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.788988035000003, 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054773589999999997, 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 676 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270571+0000", 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1199999999999999e-07 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270573+0000", 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7409999999999999e-06 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270576+0000", 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0580000000000002e-06 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270579+0000", 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0630000000000002e-06 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273029+0000", 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024502119999999998 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:49.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273139+0000", 2024-09-15T01:33:49.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110196 2024-09-15T01:33:49.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:49.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273456+0000", 2024-09-15T01:33:49.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031676000000000002 2024-09-15T01:33:49.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:49.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273543+0000", 2024-09-15T01:33:49.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7276999999999998e-05 2024-09-15T01:33:49.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:49.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273550+0000", 2024-09-15T01:33:49.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2119999999999997e-06 2024-09-15T01:33:49.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:49.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273583+0000", 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2625e-05 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273919+0000", 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033556900000000003 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325294+0000", 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051375250999999997 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325344+0000", 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0314e-05 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:677 3.5s0 3:ac3ca7d0:::smithi17529310-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.788966741000003, 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.058358134999999998, 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-15T01:33:49.143 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 677 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:49.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270592+0000", 2024-09-15T01:33:49.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.65e-07 2024-09-15T01:33:49.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270594+0000", 2024-09-15T01:33:49.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4219999999999999e-06 2024-09-15T01:33:49.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270598+0000", 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9439999999999998e-06 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.270600+0000", 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.243e-06 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.274286+0000", 2024-09-15T01:33:49.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036864089999999999 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for rw locks", 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.274337+0000", 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0677999999999998e-05 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325802+0000", 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051465576999999998 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:49.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325904+0000", 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010192 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.328950+0000", 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030455769999999998 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 71/31 ECSubWriteReply(tid=63, last_complete=71'46, committed=1, applied=1))", 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.273846+0000", 2024-09-15T01:33:49.148 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.785712191000002, 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051825606000000003, 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273846+0000", 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273846+0000", 2024-09-15T01:33:49.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273847+0000", 2024-09-15T01:33:49.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5199999999999997e-07 2024-09-15T01:33:49.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.273850+0000", 2024-09-15T01:33:49.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1690000000000001e-06 2024-09-15T01:33:49.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325190+0000", 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051340157999999997 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325198+0000", 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4839999999999998e-06 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325235+0000", 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7156000000000001e-05 2024-09-15T01:33:49.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.325672+0000", 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00043708700000000001 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 71/31 ECSubWriteReply(tid=64, last_complete=71'46, committed=1, applied=1))", 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.329874+0000", 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.729684560999999, 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.074638840999999997, 2024-09-15T01:33:49.152 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329874+0000", 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329874+0000", 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329875+0000", 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2799999999999996e-07 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.329878+0000", 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7579999999999998e-06 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404471+0000", 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.074592695000000001 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404475+0000", 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8650000000000003e-06 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404497+0000", 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2458000000000001e-05 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404513+0000", 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5537e-05 2024-09-15T01:33:49.155 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4477.0:714 3.5s0 3:a0a9f8ce:::smithi17529310-35:head [delete] snapc 22=[22,1f,1e,1d,1a,19] ondisk+write+known_if_redirected+full_force+supports_pool_eio e71)", 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.401661+0000", 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.65789771, 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044420634000000001, 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4477", 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/783755337", 2024-09-15T01:33:49.156 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 714 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401661+0000", 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401661+0000", 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401663+0000", 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401670+0000", 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3549999999999999e-06 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401687+0000", 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7388e-05 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401695+0000", 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6310000000000003e-06 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401732+0000", 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7415e-05 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.401813+0000", 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0117999999999994e-05 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402335+0000", 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052289800000000005 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402584+0000", 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024863700000000001 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402594+0000", 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7890000000000006e-06 2024-09-15T01:33:49.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402634+0000", 2024-09-15T01:33:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9984000000000002e-05 2024-09-15T01:33:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.402647+0000", 2024-09-15T01:33:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3105000000000001e-05 2024-09-15T01:33:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-15T01:33:49.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446020+0000", 2024-09-15T01:33:49.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.043373215 2024-09-15T01:33:49.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446081+0000", 2024-09-15T01:33:49.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1248999999999996e-05 2024-09-15T01:33:49.163 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.163 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.163 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.163 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 71/31 ECSubWriteReply(tid=66, last_complete=71'48, committed=1, applied=1))", 2024-09-15T01:33:49.163 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:49.163 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.655103854000004, 2024-09-15T01:33:49.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041833197000000003, 2024-09-15T01:33:49.163 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.163 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404455+0000", 2024-09-15T01:33:49.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2099999999999997e-07 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.404458+0000", 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.976e-06 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.445921+0000", 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041463284000000003 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.445925+0000", 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0260000000000001e-06 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.445975+0000", 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9424e-05 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:13.446288+0000", 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031296600000000003 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.166 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDECSubOpWrite(3.2s4 86/54 ECSubWrite(tid=305, reqid=client.4477.0:848, at_version=86'56, trim_to=0'0, roll_forward_to=85'52))", 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-15T01:33:28.135651+0000", 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.923907626999998, 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01654541, 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135651+0000", 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135651+0000", 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135651+0000", 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2000000000000001e-07 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135665+0000", 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3978000000000001e-05 2024-09-15T01:33:49.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135726+0000", 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1020000000000002e-05 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135730+0000", 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.929e-06 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135946+0000", 2024-09-15T01:33:49.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021585699999999999 2024-09-15T01:33:49.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_started", 2024-09-15T01:33:49.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.135955+0000", 2024-09-15T01:33:49.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0929999999999992e-06 2024-09-15T01:33:49.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_queued_for_journal_write", 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136046+0000", 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0948999999999999e-05 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "write_thread_in_journal_buffer", 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136096+0000", 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0049999999999997e-05 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "journaled_completion_queued", 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136268+0000", 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000172127 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_committed", 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.136324+0000", 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5379e-05 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-15T01:33:28.152196+0000", 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015872807999999999 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-15T01:33:49.172 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-15T01:33:49.174 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T01:33:49.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-15T01:33:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-15T01:33:50.169 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T01:33:51.156 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T01:33:51.173 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-15T01:33:52.164 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-15T01:33:52.181 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-15T01:33:53.178 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.8 to 1 (10000) 2024-09-15T01:33:53.194 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 1 2024-09-15T01:33:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-09-15T01:33:54.199 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-09-15T01:33:55.188 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.12 to 1 (10000) 2024-09-15T01:33:55.205 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 12 1 2024-09-15T01:33:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.12 primary-affinity to 1 (8655362) 2024-09-15T01:33:56.210 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T01:33:57.199 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T01:33:57.217 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-15T01:33:58.204 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-15T01:33:58.221 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T01:33:59.209 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T01:33:59.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-15T01:34:00.227 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-15T01:34:00.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-09-15T01:34:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.9 to 1 (10000) 2024-09-15T01:34:01.251 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 9 1 2024-09-15T01:34:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-09-15T01:34:02.259 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-09-15T01:34:03.248 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.13 to 1 (10000) 2024-09-15T01:34:03.264 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 13 1 2024-09-15T01:34:04.254 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.13 primary-affinity to 1 (8655362) 2024-09-15T01:34:04.271 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T01:34:05.259 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T01:34:05.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-15T01:34:06.267 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-15T01:34:06.283 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-15T01:34:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-15T01:34:07.289 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-15T01:34:08.278 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-15T01:34:08.295 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-15T01:34:09.283 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.10 to 1 (10000) 2024-09-15T01:34:09.299 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 10 1 2024-09-15T01:34:10.291 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-09-15T01:34:10.308 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-09-15T01:34:11.304 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.14 to 1 (10000) 2024-09-15T01:34:11.320 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 14 1 2024-09-15T01:34:12.305 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.14 primary-affinity to 1 (8655362) 2024-09-15T01:34:12.321 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T01:34:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T01:34:13.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-15T01:34:14.315 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-15T01:34:14.332 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-15T01:34:15.319 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-15T01:34:15.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-15T01:34:16.327 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-15T01:34:16.343 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-15T01:34:17.332 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.11 to 1 (10000) 2024-09-15T01:34:17.347 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 11 1 2024-09-15T01:34:18.337 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-09-15T01:34:18.352 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-09-15T01:34:19.342 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.15 to 1 (10000) 2024-09-15T01:34:19.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 15 1 2024-09-15T01:34:20.349 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.15 primary-affinity to 1 (8655362) 2024-09-15T01:34:20.366 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-15T01:34:20.366 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-15T01:34:20.366 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-15T01:34:20.366 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-15T01:34:20.366 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T01:34:20.367 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:34:20.657 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:34:20.658 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":137,"fsid":"d4803586-ed44-4154-8ee1-6b95ec3e17e7","created":"2024-09-15T01:32:07.285177+0000","modified":"2024-09-15T01:34:20.344742+0000","last_up_change":"2024-09-15T01:32:14.443323+0000","last_in_change":"2024-09-15T01:33:08.068993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:32:15.118634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:32:21.824131+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bec41583-a0a4-4712-a9ec-eec1e88fb60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":28861},{"type":"v1","addr":"172.21.15.42:6809","nonce":28861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":28861},{"type":"v1","addr":"172.21.15.42:6811","nonce":28861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":28861},{"type":"v1","addr":"172.21.15.42:6815","nonce":28861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":28861},{"type":"v1","addr":"172.21.15.42:6813","nonce":28861}]},"public_addr":"172.21.15.42:6809/28861","cluster_addr":"172.21.15.42:6811/28861","heartbeat_back_addr":"172.21.15.42:6815/28861","heartbeat_front_addr":"172.21.15.42:6813/28861","state":["exists","up"]},{"osd":1,"uuid":"2cca66be-8685-4a38-abf8-c864cd5434da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":28508},{"type":"v1","addr":"172.21.15.115:6809","nonce":28508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":28508},{"type":"v1","addr":"172.21.15.115:6811","nonce":28508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6817","nonce":28508},{"type":"v1","addr":"172.21.15.115:6819","nonce":28508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6813","nonce":28508},{"type":"v1","addr":"172.21.15.115:6815","nonce":28508}]},"public_addr":"172.21.15.115:6809/28508","cluster_addr":"172.21.15.115:6811/28508","heartbeat_back_addr":"172.21.15.115:6819/28508","heartbeat_front_addr":"172.21.15.115:6815/28508","state":["exists","up"]},{"osd":2,"uuid":"b8756aac-3a3e-41ba-a6ef-497063cb44df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":28732},{"type":"v1","addr":"172.21.15.152:6809","nonce":28732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":28732},{"type":"v1","addr":"172.21.15.152:6811","nonce":28732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":28732},{"type":"v1","addr":"172.21.15.152:6816","nonce":28732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":28732},{"type":"v1","addr":"172.21.15.152:6813","nonce":28732}]},"public_addr":"172.21.15.152:6809/28732","cluster_addr":"172.21.15.152:6811/28732","heartbeat_back_addr":"172.21.15.152:6816/28732","heartbeat_front_addr":"172.21.15.152:6813/28732","state":["exists","up"]},{"osd":3,"uuid":"554875f5-74c0-4b58-80ee-6ba93748f104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":28835},{"type":"v1","addr":"172.21.15.175:6803","nonce":28835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":28835},{"type":"v1","addr":"172.21.15.175:6805","nonce":28835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":28835},{"type":"v1","addr":"172.21.15.175:6809","nonce":28835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":28835},{"type":"v1","addr":"172.21.15.175:6807","nonce":28835}]},"public_addr":"172.21.15.175:6803/28835","cluster_addr":"172.21.15.175:6805/28835","heartbeat_back_addr":"172.21.15.175:6809/28835","heartbeat_front_addr":"172.21.15.175:6807/28835","state":["exists","up"]},{"osd":4,"uuid":"3fe2c05a-4c22-4dd7-a8c6-282155c82eb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":28860},{"type":"v1","addr":"172.21.15.42:6801","nonce":28860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":28860},{"type":"v1","addr":"172.21.15.42:6803","nonce":28860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":28860},{"type":"v1","addr":"172.21.15.42:6807","nonce":28860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":28860},{"type":"v1","addr":"172.21.15.42:6805","nonce":28860}]},"public_addr":"172.21.15.42:6801/28860","cluster_addr":"172.21.15.42:6803/28860","heartbeat_back_addr":"172.21.15.42:6807/28860","heartbeat_front_addr":"172.21.15.42:6805/28860","state":["exists","up"]},{"osd":5,"uuid":"a301d1ce-3e31-4782-ade3-f9c0f1e6a95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":28509},{"type":"v1","addr":"172.21.15.115:6801","nonce":28509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":28509},{"type":"v1","addr":"172.21.15.115:6803","nonce":28509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":28509},{"type":"v1","addr":"172.21.15.115:6807","nonce":28509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":28509},{"type":"v1","addr":"172.21.15.115:6805","nonce":28509}]},"public_addr":"172.21.15.115:6801/28509","cluster_addr":"172.21.15.115:6803/28509","heartbeat_back_addr":"172.21.15.115:6807/28509","heartbeat_front_addr":"172.21.15.115:6805/28509","state":["exists","up"]},{"osd":6,"uuid":"c7d180be-21c5-4674-a9f2-10a1c7794871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":28731},{"type":"v1","addr":"172.21.15.152:6801","nonce":28731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":28731},{"type":"v1","addr":"172.21.15.152:6803","nonce":28731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":28731},{"type":"v1","addr":"172.21.15.152:6807","nonce":28731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":28731},{"type":"v1","addr":"172.21.15.152:6805","nonce":28731}]},"public_addr":"172.21.15.152:6801/28731","cluster_addr":"172.21.15.152:6803/28731","heartbeat_back_addr":"172.21.15.152:6807/28731","heartbeat_front_addr":"172.21.15.152:6805/28731","state":["exists","up"]},{"osd":7,"uuid":"4d2429ea-772f-4fb9-a421-240f1399f16c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":28836},{"type":"v1","addr":"172.21.15.175:6811","nonce":28836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":28836},{"type":"v1","addr":"172.21.15.175:6813","nonce":28836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":28836},{"type":"v1","addr":"172.21.15.175:6817","nonce":28836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":28836},{"type":"v1","addr":"172.21.15.175:6815","nonce":28836}]},"public_addr":"172.21.15.175:6811/28836","cluster_addr":"172.21.15.175:6813/28836","heartbeat_back_addr":"172.21.15.175:6817/28836","heartbeat_front_addr":"172.21.15.175:6815/28836","state":["exists","up"]},{"osd":8,"uuid":"de7389ab-a29e-4177-9e94-484fc6a94102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":28862},{"type":"v1","addr":"172.21.15.42:6817","nonce":28862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":28862},{"type":"v1","addr":"172.21.15.42:6819","nonce":28862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":28862},{"type":"v1","addr":"172.21.15.42:6823","nonce":28862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":28862},{"type":"v1","addr":"172.21.15.42:6821","nonce":28862}]},"public_addr":"172.21.15.42:6817/28862","cluster_addr":"172.21.15.42:6819/28862","heartbeat_back_addr":"172.21.15.42:6823/28862","heartbeat_front_addr":"172.21.15.42:6821/28862","state":["exists","up"]},{"osd":9,"uuid":"4b4b44da-97d6-41f2-8fc6-57680b10aeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":28510},{"type":"v1","addr":"172.21.15.115:6814","nonce":28510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":28510},{"type":"v1","addr":"172.21.15.115:6818","nonce":28510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":28510},{"type":"v1","addr":"172.21.15.115:6823","nonce":28510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":28510},{"type":"v1","addr":"172.21.15.115:6821","nonce":28510}]},"public_addr":"172.21.15.115:6814/28510","cluster_addr":"172.21.15.115:6818/28510","heartbeat_back_addr":"172.21.15.115:6823/28510","heartbeat_front_addr":"172.21.15.115:6821/28510","state":["exists","up"]},{"osd":10,"uuid":"5f285a07-9057-4575-b911-b3977acd397c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":28733},{"type":"v1","addr":"172.21.15.152:6817","nonce":28733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":28733},{"type":"v1","addr":"172.21.15.152:6819","nonce":28733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":28733},{"type":"v1","addr":"172.21.15.152:6824","nonce":28733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":28733},{"type":"v1","addr":"172.21.15.152:6821","nonce":28733}]},"public_addr":"172.21.15.152:6817/28733","cluster_addr":"172.21.15.152:6819/28733","heartbeat_back_addr":"172.21.15.152:6824/28733","heartbeat_front_addr":"172.21.15.152:6821/28733","state":["exists","up"]},{"osd":11,"uuid":"dacc0f3f-a598-4055-92f4-d41102ceca83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":28837},{"type":"v1","addr":"172.21.15.175:6827","nonce":28837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":28837},{"type":"v1","addr":"172.21.15.175:6829","nonce":28837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":28837},{"type":"v1","addr":"172.21.15.175:6833","nonce":28837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":28837},{"type":"v1","addr":"172.21.15.175:6831","nonce":28837}]},"public_addr":"172.21.15.175:6827/28837","cluster_addr":"172.21.15.175:6829/28837","heartbeat_back_addr":"172.21.15.175:6833/28837","heartbeat_front_addr":"172.21.15.175:6831/28837","state":["exists","up"]},{"osd":12,"uuid":"d218197b-d0db-410c-abff-8b849a21a555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":28866},{"type":"v1","addr":"172.21.15.42:6825","nonce":28866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":28866},{"type":"v1","addr":"172.21.15.42:6827","nonce":28866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":28866},{"type":"v1","addr":"172.21.15.42:6831","nonce":28866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":28866},{"type":"v1","addr":"172.21.15.42:6829","nonce":28866}]},"public_addr":"172.21.15.42:6825/28866","cluster_addr":"172.21.15.42:6827/28866","heartbeat_back_addr":"172.21.15.42:6831/28866","heartbeat_front_addr":"172.21.15.42:6829/28866","state":["exists","up"]},{"osd":13,"uuid":"c9bf5faf-f2ee-482e-b903-28c7718e2063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":28511},{"type":"v1","addr":"172.21.15.115:6825","nonce":28511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":28511},{"type":"v1","addr":"172.21.15.115:6827","nonce":28511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":28511},{"type":"v1","addr":"172.21.15.115:6831","nonce":28511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":28511},{"type":"v1","addr":"172.21.15.115:6829","nonce":28511}]},"public_addr":"172.21.15.115:6825/28511","cluster_addr":"172.21.15.115:6827/28511","heartbeat_back_addr":"172.21.15.115:6831/28511","heartbeat_front_addr":"172.21.15.115:6829/28511","state":["exists","up"]},{"osd":14,"uuid":"cc3f19cf-5a97-440b-a953-7727e6ed6a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6823","nonce":28734},{"type":"v1","addr":"172.21.15.152:6825","nonce":28734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":28734},{"type":"v1","addr":"172.21.15.152:6827","nonce":28734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":28734},{"type":"v1","addr":"172.21.15.152:6831","nonce":28734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":28734},{"type":"v1","addr":"172.21.15.152:6829","nonce":28734}]},"public_addr":"172.21.15.152:6825/28734","cluster_addr":"172.21.15.152:6827/28734","heartbeat_back_addr":"172.21.15.152:6831/28734","heartbeat_front_addr":"172.21.15.152:6829/28734","state":["exists","up"]},{"osd":15,"uuid":"51290c23-e86d-4e05-9ac8-ca53d6a1eb80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":28838},{"type":"v1","addr":"172.21.15.175:6819","nonce":28838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":28838},{"type":"v1","addr":"172.21.15.175:6821","nonce":28838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":28838},{"type":"v1","addr":"172.21.15.175:6825","nonce":28838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":28838},{"type":"v1","addr":"172.21.15.175:6823","nonce":28838}]},"public_addr":"172.21.15.175:6819/28838","cluster_addr":"172.21.15.175:6821/28838","heartbeat_back_addr":"172.21.15.175:6825/28838","heartbeat_front_addr":"172.21.15.175:6823/28838","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:15.503649+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:25.772185+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:32.378075+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:25.443004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:23.634325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.175620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.608100+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.434930+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:20.501276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:13.348208+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.326775+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.748721+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.563588+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:27.752979+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:19.554554+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:32.885128+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[10,8]}],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":25,"length":1},{"begin":31,"length":1},{"begin":41,"length":1},{"begin":44,"length":1},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":58,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:34:20.670 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T01:34:20.671 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:34:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:34:20.949 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":137,"fsid":"d4803586-ed44-4154-8ee1-6b95ec3e17e7","created":"2024-09-15T01:32:07.285177+0000","modified":"2024-09-15T01:34:20.344742+0000","last_up_change":"2024-09-15T01:32:14.443323+0000","last_in_change":"2024-09-15T01:33:08.068993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:32:15.118634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:32:21.824131+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bec41583-a0a4-4712-a9ec-eec1e88fb60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":28861},{"type":"v1","addr":"172.21.15.42:6809","nonce":28861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":28861},{"type":"v1","addr":"172.21.15.42:6811","nonce":28861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":28861},{"type":"v1","addr":"172.21.15.42:6815","nonce":28861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":28861},{"type":"v1","addr":"172.21.15.42:6813","nonce":28861}]},"public_addr":"172.21.15.42:6809/28861","cluster_addr":"172.21.15.42:6811/28861","heartbeat_back_addr":"172.21.15.42:6815/28861","heartbeat_front_addr":"172.21.15.42:6813/28861","state":["exists","up"]},{"osd":1,"uuid":"2cca66be-8685-4a38-abf8-c864cd5434da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":28508},{"type":"v1","addr":"172.21.15.115:6809","nonce":28508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":28508},{"type":"v1","addr":"172.21.15.115:6811","nonce":28508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6817","nonce":28508},{"type":"v1","addr":"172.21.15.115:6819","nonce":28508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6813","nonce":28508},{"type":"v1","addr":"172.21.15.115:6815","nonce":28508}]},"public_addr":"172.21.15.115:6809/28508","cluster_addr":"172.21.15.115:6811/28508","heartbeat_back_addr":"172.21.15.115:6819/28508","heartbeat_front_addr":"172.21.15.115:6815/28508","state":["exists","up"]},{"osd":2,"uuid":"b8756aac-3a3e-41ba-a6ef-497063cb44df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":28732},{"type":"v1","addr":"172.21.15.152:6809","nonce":28732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":28732},{"type":"v1","addr":"172.21.15.152:6811","nonce":28732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":28732},{"type":"v1","addr":"172.21.15.152:6816","nonce":28732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":28732},{"type":"v1","addr":"172.21.15.152:6813","nonce":28732}]},"public_addr":"172.21.15.152:6809/28732","cluster_addr":"172.21.15.152:6811/28732","heartbeat_back_addr":"172.21.15.152:6816/28732","heartbeat_front_addr":"172.21.15.152:6813/28732","state":["exists","up"]},{"osd":3,"uuid":"554875f5-74c0-4b58-80ee-6ba93748f104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":28835},{"type":"v1","addr":"172.21.15.175:6803","nonce":28835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":28835},{"type":"v1","addr":"172.21.15.175:6805","nonce":28835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":28835},{"type":"v1","addr":"172.21.15.175:6809","nonce":28835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":28835},{"type":"v1","addr":"172.21.15.175:6807","nonce":28835}]},"public_addr":"172.21.15.175:6803/28835","cluster_addr":"172.21.15.175:6805/28835","heartbeat_back_addr":"172.21.15.175:6809/28835","heartbeat_front_addr":"172.21.15.175:6807/28835","state":["exists","up"]},{"osd":4,"uuid":"3fe2c05a-4c22-4dd7-a8c6-282155c82eb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":28860},{"type":"v1","addr":"172.21.15.42:6801","nonce":28860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":28860},{"type":"v1","addr":"172.21.15.42:6803","nonce":28860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":28860},{"type":"v1","addr":"172.21.15.42:6807","nonce":28860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":28860},{"type":"v1","addr":"172.21.15.42:6805","nonce":28860}]},"public_addr":"172.21.15.42:6801/28860","cluster_addr":"172.21.15.42:6803/28860","heartbeat_back_addr":"172.21.15.42:6807/28860","heartbeat_front_addr":"172.21.15.42:6805/28860","state":["exists","up"]},{"osd":5,"uuid":"a301d1ce-3e31-4782-ade3-f9c0f1e6a95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":28509},{"type":"v1","addr":"172.21.15.115:6801","nonce":28509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":28509},{"type":"v1","addr":"172.21.15.115:6803","nonce":28509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":28509},{"type":"v1","addr":"172.21.15.115:6807","nonce":28509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":28509},{"type":"v1","addr":"172.21.15.115:6805","nonce":28509}]},"public_addr":"172.21.15.115:6801/28509","cluster_addr":"172.21.15.115:6803/28509","heartbeat_back_addr":"172.21.15.115:6807/28509","heartbeat_front_addr":"172.21.15.115:6805/28509","state":["exists","up"]},{"osd":6,"uuid":"c7d180be-21c5-4674-a9f2-10a1c7794871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":28731},{"type":"v1","addr":"172.21.15.152:6801","nonce":28731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":28731},{"type":"v1","addr":"172.21.15.152:6803","nonce":28731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":28731},{"type":"v1","addr":"172.21.15.152:6807","nonce":28731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":28731},{"type":"v1","addr":"172.21.15.152:6805","nonce":28731}]},"public_addr":"172.21.15.152:6801/28731","cluster_addr":"172.21.15.152:6803/28731","heartbeat_back_addr":"172.21.15.152:6807/28731","heartbeat_front_addr":"172.21.15.152:6805/28731","state":["exists","up"]},{"osd":7,"uuid":"4d2429ea-772f-4fb9-a421-240f1399f16c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":28836},{"type":"v1","addr":"172.21.15.175:6811","nonce":28836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":28836},{"type":"v1","addr":"172.21.15.175:6813","nonce":28836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":28836},{"type":"v1","addr":"172.21.15.175:6817","nonce":28836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":28836},{"type":"v1","addr":"172.21.15.175:6815","nonce":28836}]},"public_addr":"172.21.15.175:6811/28836","cluster_addr":"172.21.15.175:6813/28836","heartbeat_back_addr":"172.21.15.175:6817/28836","heartbeat_front_addr":"172.21.15.175:6815/28836","state":["exists","up"]},{"osd":8,"uuid":"de7389ab-a29e-4177-9e94-484fc6a94102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":28862},{"type":"v1","addr":"172.21.15.42:6817","nonce":28862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":28862},{"type":"v1","addr":"172.21.15.42:6819","nonce":28862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":28862},{"type":"v1","addr":"172.21.15.42:6823","nonce":28862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":28862},{"type":"v1","addr":"172.21.15.42:6821","nonce":28862}]},"public_addr":"172.21.15.42:6817/28862","cluster_addr":"172.21.15.42:6819/28862","heartbeat_back_addr":"172.21.15.42:6823/28862","heartbeat_front_addr":"172.21.15.42:6821/28862","state":["exists","up"]},{"osd":9,"uuid":"4b4b44da-97d6-41f2-8fc6-57680b10aeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":28510},{"type":"v1","addr":"172.21.15.115:6814","nonce":28510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":28510},{"type":"v1","addr":"172.21.15.115:6818","nonce":28510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":28510},{"type":"v1","addr":"172.21.15.115:6823","nonce":28510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":28510},{"type":"v1","addr":"172.21.15.115:6821","nonce":28510}]},"public_addr":"172.21.15.115:6814/28510","cluster_addr":"172.21.15.115:6818/28510","heartbeat_back_addr":"172.21.15.115:6823/28510","heartbeat_front_addr":"172.21.15.115:6821/28510","state":["exists","up"]},{"osd":10,"uuid":"5f285a07-9057-4575-b911-b3977acd397c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":28733},{"type":"v1","addr":"172.21.15.152:6817","nonce":28733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":28733},{"type":"v1","addr":"172.21.15.152:6819","nonce":28733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":28733},{"type":"v1","addr":"172.21.15.152:6824","nonce":28733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":28733},{"type":"v1","addr":"172.21.15.152:6821","nonce":28733}]},"public_addr":"172.21.15.152:6817/28733","cluster_addr":"172.21.15.152:6819/28733","heartbeat_back_addr":"172.21.15.152:6824/28733","heartbeat_front_addr":"172.21.15.152:6821/28733","state":["exists","up"]},{"osd":11,"uuid":"dacc0f3f-a598-4055-92f4-d41102ceca83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":28837},{"type":"v1","addr":"172.21.15.175:6827","nonce":28837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":28837},{"type":"v1","addr":"172.21.15.175:6829","nonce":28837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":28837},{"type":"v1","addr":"172.21.15.175:6833","nonce":28837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":28837},{"type":"v1","addr":"172.21.15.175:6831","nonce":28837}]},"public_addr":"172.21.15.175:6827/28837","cluster_addr":"172.21.15.175:6829/28837","heartbeat_back_addr":"172.21.15.175:6833/28837","heartbeat_front_addr":"172.21.15.175:6831/28837","state":["exists","up"]},{"osd":12,"uuid":"d218197b-d0db-410c-abff-8b849a21a555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":28866},{"type":"v1","addr":"172.21.15.42:6825","nonce":28866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":28866},{"type":"v1","addr":"172.21.15.42:6827","nonce":28866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":28866},{"type":"v1","addr":"172.21.15.42:6831","nonce":28866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":28866},{"type":"v1","addr":"172.21.15.42:6829","nonce":28866}]},"public_addr":"172.21.15.42:6825/28866","cluster_addr":"172.21.15.42:6827/28866","heartbeat_back_addr":"172.21.15.42:6831/28866","heartbeat_front_addr":"172.21.15.42:6829/28866","state":["exists","up"]},{"osd":13,"uuid":"c9bf5faf-f2ee-482e-b903-28c7718e2063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":28511},{"type":"v1","addr":"172.21.15.115:6825","nonce":28511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":28511},{"type":"v1","addr":"172.21.15.115:6827","nonce":28511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":28511},{"type":"v1","addr":"172.21.15.115:6831","nonce":28511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":28511},{"type":"v1","addr":"172.21.15.115:6829","nonce":28511}]},"public_addr":"172.21.15.115:6825/28511","cluster_addr":"172.21.15.115:6827/28511","heartbeat_back_addr":"172.21.15.115:6831/28511","heartbeat_front_addr":"172.21.15.115:6829/28511","state":["exists","up"]},{"osd":14,"uuid":"cc3f19cf-5a97-440b-a953-7727e6ed6a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6823","nonce":28734},{"type":"v1","addr":"172.21.15.152:6825","nonce":28734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":28734},{"type":"v1","addr":"172.21.15.152:6827","nonce":28734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":28734},{"type":"v1","addr":"172.21.15.152:6831","nonce":28734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":28734},{"type":"v1","addr":"172.21.15.152:6829","nonce":28734}]},"public_addr":"172.21.15.152:6825/28734","cluster_addr":"172.21.15.152:6827/28734","heartbeat_back_addr":"172.21.15.152:6831/28734","heartbeat_front_addr":"172.21.15.152:6829/28734","state":["exists","up"]},{"osd":15,"uuid":"51290c23-e86d-4e05-9ac8-ca53d6a1eb80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":28838},{"type":"v1","addr":"172.21.15.175:6819","nonce":28838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":28838},{"type":"v1","addr":"172.21.15.175:6821","nonce":28838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":28838},{"type":"v1","addr":"172.21.15.175:6825","nonce":28838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":28838},{"type":"v1","addr":"172.21.15.175:6823","nonce":28838}]},"public_addr":"172.21.15.175:6819/28838","cluster_addr":"172.21.15.175:6821/28838","heartbeat_back_addr":"172.21.15.175:6825/28838","heartbeat_front_addr":"172.21.15.175:6823/28838","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:15.503649+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:25.772185+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:32.378075+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:25.443004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:23.634325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.175620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.608100+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.434930+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:20.501276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:13.348208+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.326775+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.748721+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.563588+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:27.752979+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:19.554554+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:32.885128+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[10,8]}],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":25,"length":1},{"begin":31,"length":1},{"begin":41,"length":1},{"begin":44,"length":1},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":58,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:34:20.962 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T01:34:20.962 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T01:34:20.962 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T01:34:20.963 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T01:34:20.963 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T01:34:20.964 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T01:34:20.964 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T01:34:20.964 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T01:34:21.006 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-15T01:34:21.006 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-15T01:34:21.007 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-15T01:34:21.007 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-15T01:34:21.007 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-09-15T01:34:21.008 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-09-15T01:34:21.008 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-09-15T01:34:21.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-09-15T01:34:21.378 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:21.379 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T01:34:21.452 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:21.452 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-15T01:34:21.453 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:21.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-15T01:34:21.454 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:21.454 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T01:34:21.464 INFO:teuthology.orchestra.run.smithi042.stdout:90194313246 2024-09-15T01:34:21.464 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T01:34:21.482 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:21.482 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:21.483 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T01:34:21.483 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:21.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T01:34:21.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T01:34:21.485 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:21.486 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T01:34:21.492 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:21.492 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-15T01:34:21.504 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:21.504 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-15T01:34:21.517 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:21.517 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:21.517 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T01:34:21.518 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:21.518 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-15T01:34:21.519 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-15T01:34:21.520 INFO:teuthology.orchestra.run.smithi042.stdout:90194313247 2024-09-15T01:34:21.520 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-15T01:34:21.532 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:21.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-15T01:34:22.319 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:22.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313246 got 90194313245 for osd.0 2024-09-15T01:34:22.361 INFO:teuthology.orchestra.run.smithi042.stdout:90194313243 2024-09-15T01:34:22.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313243 for osd.6 2024-09-15T01:34:22.457 INFO:teuthology.orchestra.run.smithi042.stdout:90194313243 2024-09-15T01:34:22.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313243 for osd.5 2024-09-15T01:34:22.517 INFO:teuthology.orchestra.run.smithi042.stdout:90194313243 2024-09-15T01:34:22.522 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:22.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313244 for osd.13 2024-09-15T01:34:22.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313243 for osd.11 2024-09-15T01:34:22.590 INFO:teuthology.orchestra.run.smithi042.stdout:90194313243 2024-09-15T01:34:22.592 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:22.594 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:22.616 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:22.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313244 for osd.7 2024-09-15T01:34:22.638 INFO:teuthology.orchestra.run.smithi042.stdout:90194313246 2024-09-15T01:34:22.642 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313243 for osd.10 2024-09-15T01:34:22.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313244 for osd.15 2024-09-15T01:34:22.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313244 for osd.3 2024-09-15T01:34:22.646 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:22.648 INFO:teuthology.orchestra.run.smithi042.stdout:90194313243 2024-09-15T01:34:22.648 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:22.651 INFO:teuthology.orchestra.run.smithi042.stdout:90194313243 2024-09-15T01:34:22.660 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:22.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313246 for osd.14 2024-09-15T01:34:22.668 INFO:teuthology.orchestra.run.smithi042.stdout:90194313243 2024-09-15T01:34:22.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313243 for osd.2 2024-09-15T01:34:22.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313244 for osd.4 2024-09-15T01:34:22.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313244 for osd.12 2024-09-15T01:34:22.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313243 for osd.9 2024-09-15T01:34:22.677 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313244 for osd.1 2024-09-15T01:34:22.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313243 for osd.8 2024-09-15T01:34:23.353 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T01:34:23.386 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T01:34:23.502 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T01:34:23.554 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-15T01:34:23.556 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-15T01:34:23.622 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T01:34:23.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-15T01:34:23.643 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-15T01:34:23.644 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T01:34:23.665 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-15T01:34:23.670 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T01:34:23.672 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T01:34:23.673 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-15T01:34:23.676 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-15T01:34:23.678 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T01:34:23.686 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi042.stdout:90194313246 2024-09-15T01:34:23.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313246 got 90194313246 for osd.0 2024-09-15T01:34:23.911 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:24.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313244 for osd.6 2024-09-15T01:34:24.041 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:24.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313244 for osd.5 2024-09-15T01:34:24.208 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.210 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:24.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313244 for osd.11 2024-09-15T01:34:24.245 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:24.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313245 for osd.13 2024-09-15T01:34:24.351 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.397 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:24.415 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313245 for osd.7 2024-09-15T01:34:24.415 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.448 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:24.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313244 for osd.10 2024-09-15T01:34:24.494 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.540 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:24.585 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313245 for osd.15 2024-09-15T01:34:24.585 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.598 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:24.605 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313245 for osd.12 2024-09-15T01:34:24.605 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.606 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:24.608 INFO:teuthology.orchestra.run.smithi042.stdout:90194313247 2024-09-15T01:34:24.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313244 for osd.9 2024-09-15T01:34:24.614 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313247 for osd.14 2024-09-15T01:34:24.627 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313245 for osd.3 2024-09-15T01:34:24.627 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.632 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:24.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313244 for osd.8 2024-09-15T01:34:24.646 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.656 INFO:teuthology.orchestra.run.smithi042.stdout:90194313244 2024-09-15T01:34:24.659 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:24.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313245 for osd.1 2024-09-15T01:34:24.672 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313244 for osd.2 2024-09-15T01:34:24.673 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.689 INFO:teuthology.orchestra.run.smithi042.stdout:90194313245 2024-09-15T01:34:24.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313245 got 90194313245 for osd.4 2024-09-15T01:34:24.702 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:24.702 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T01:34:24.702 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:34:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:34:25.023 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:34:25.033 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-15T01:34:23.095099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":5,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":12773916,"kb_used_data":1486100804,"kb_used_omap":52110,"kb_used_meta":0,"kb_avail":1486099940,"statfs":{"total":1534846828544,"available":1521766338560,"internally_reserved":884736,"allocated":1521767223296,"data_stored":1521767223296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53361491,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.760385"},"pg_stats":[{"pgid":"1.0","version":"106'114","reported_seq":266,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888285+0000","last_change":"2024-09-15T01:33:49.902092+0000","last_active":"2024-09-15T01:34:19.888285+0000","last_peered":"2024-09-15T01:34:19.888285+0000","last_clean":"2024-09-15T01:34:19.888285+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:34:19.888285+0000","last_undegraded":"2024-09-15T01:34:19.888285+0000","last_fullsized":"2024-09-15T01:34:19.888285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'114","last_scrub_stamp":"2024-09-15T01:33:49.902036+0000","last_deep_scrub":"106'114","last_deep_scrub_stamp":"2024-09-15T01:33:49.902036+0000","last_clean_scrub_stamp":"2024-09-15T01:33:49.902036+0000","objects_scrubbed":2,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:12.193331+0000","scrub_duration":0.010874557,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"105'2","reported_seq":128,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.341551+0000","last_change":"2024-09-15T01:33:48.491848+0000","last_active":"2024-09-15T01:34:19.341551+0000","last_peered":"2024-09-15T01:34:19.341551+0000","last_clean":"2024-09-15T01:34:19.341551+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:34:19.341551+0000","last_undegraded":"2024-09-15T01:34:19.341551+0000","last_fullsized":"2024-09-15T01:34:19.341551+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'2","last_scrub_stamp":"2024-09-15T01:33:48.491771+0000","last_deep_scrub":"105'2","last_deep_scrub_stamp":"2024-09-15T01:33:48.491771+0000","last_clean_scrub_stamp":"2024-09-15T01:33:48.491771+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:01.566416+0000","scrub_duration":0.0025105769999999999,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.794589+0000","last_change":"2024-09-15T01:33:46.379860+0000","last_active":"2024-09-15T01:34:20.794589+0000","last_peered":"2024-09-15T01:34:20.794589+0000","last_clean":"2024-09-15T01:34:20.794589+0000","last_became_active":"2024-09-15T01:33:23.083242+0000","last_became_peered":"2024-09-15T01:33:23.083242+0000","last_unstale":"2024-09-15T01:34:20.794589+0000","last_undegraded":"2024-09-15T01:34:20.794589+0000","last_fullsized":"2024-09-15T01:34:20.794589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:46.379809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:46.379809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:49.087667+0000","scrub_duration":0.001946382,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8],"acting":[10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888248+0000","last_change":"2024-09-15T01:33:50.941845+0000","last_active":"2024-09-15T01:34:19.888248+0000","last_peered":"2024-09-15T01:34:19.888248+0000","last_clean":"2024-09-15T01:34:19.888248+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:34:19.888248+0000","last_undegraded":"2024-09-15T01:34:19.888248+0000","last_fullsized":"2024-09-15T01:34:19.888248+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:50.941790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:50.941790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:10.537027+0000","scrub_duration":0.0017165430000000001,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":133,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.484672+0000","last_change":"2024-09-15T01:33:46.870657+0000","last_active":"2024-09-15T01:34:19.484672+0000","last_peered":"2024-09-15T01:34:19.484672+0000","last_clean":"2024-09-15T01:34:19.484672+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:34:19.484672+0000","last_undegraded":"2024-09-15T01:34:19.484672+0000","last_fullsized":"2024-09-15T01:34:19.484672+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-15T01:33:46.870584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:46.870584+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:14.306792+0000","scrub_duration":0.0027185820000000002,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888035+0000","last_change":"2024-09-15T01:33:41.548849+0000","last_active":"2024-09-15T01:34:19.888035+0000","last_peered":"2024-09-15T01:34:19.888035+0000","last_clean":"2024-09-15T01:34:19.888035+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:34:19.888035+0000","last_undegraded":"2024-09-15T01:34:19.888035+0000","last_fullsized":"2024-09-15T01:34:19.888035+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:41.548799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:41.548799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:55.947522+0000","scrub_duration":0.001258681,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888166+0000","last_change":"2024-09-15T01:33:41.543896+0000","last_active":"2024-09-15T01:34:19.888166+0000","last_peered":"2024-09-15T01:34:19.888166+0000","last_clean":"2024-09-15T01:34:19.888166+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:34:19.888166+0000","last_undegraded":"2024-09-15T01:34:19.888166+0000","last_fullsized":"2024-09-15T01:34:19.888166+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:41.543850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:41.543850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:56.945346+0000","scrub_duration":0.0015955419999999999,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.587067+0000","last_change":"2024-09-15T01:33:39.638452+0000","last_active":"2024-09-15T01:34:20.587067+0000","last_peered":"2024-09-15T01:34:20.587067+0000","last_clean":"2024-09-15T01:34:20.587067+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:34:20.587067+0000","last_undegraded":"2024-09-15T01:34:20.587067+0000","last_fullsized":"2024-09-15T01:34:20.587067+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:39.638404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:39.638404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:59.511423+0000","scrub_duration":0.001362835,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888494+0000","last_change":"2024-09-15T01:33:38.175676+0000","last_active":"2024-09-15T01:34:19.888494+0000","last_peered":"2024-09-15T01:34:19.888494+0000","last_clean":"2024-09-15T01:34:19.888494+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:34:19.888494+0000","last_undegraded":"2024-09-15T01:34:19.888494+0000","last_fullsized":"2024-09-15T01:34:19.888494+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:38.175625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:38.175625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:44.144155+0000","scrub_duration":0.001339526,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881456,"kb_used_omap":3954,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":65536,"allocated":95110610944,"data_stored":95110610944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4049435,"internal_metadata":0},"hb_peers":[1,2,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":16.196999999999999,"5min":27.753,"15min":29.678999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":299.721,"5min":574.26900000000001,"15min":574.26900000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":18.404,"5min":4.0270000000000001,"15min":1.631},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":342.08300000000003,"5min":342.08300000000003,"15min":342.08300000000003},"last":0.35799999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:33:20 2024","interfaces":[{"interface":"back","average":{"1min":32.810000000000002,"5min":32.810000000000002,"15min":32.810000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":644.97699999999998,"5min":644.97699999999998,"15min":644.97699999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":805.39499999999998}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.71399999999999997,"15min":0.73999999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.69299999999999995,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":11.887,"5min":2.8730000000000002,"15min":1.371},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":214.994,"5min":214.994,"15min":214.994},"last":0.66900000000000004}]},{"osd":5,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47699999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.502}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82899999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.621},{"interface":"front","average":{"1min":10.272,"5min":2.625,"15min":1.351},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":173.11099999999999,"5min":173.11099999999999,"15min":173.11099999999999},"last":0.48599999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":6.056,"15min":6.9420000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.589,"5min":134.12200000000001,"15min":134.12200000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":45.521000000000001,"5min":9.7249999999999996,"15min":3.7589999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":846.97400000000005,"5min":846.97400000000005,"15min":846.97400000000005},"last":0.65400000000000003}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.56599999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.5649999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":17.137,"5min":4.0940000000000003,"15min":1.9199999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":313.495,"5min":313.495,"15min":313.495},"last":0.72499999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.69399999999999995}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81399999999999995,"15min":0.81299999999999994},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.70899999999999996}]}]},{"osd":9,"up_from":21,"seq":90194313244,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881452,"kb_used_omap":1548,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":61440,"allocated":95110606848,"data_stored":95110606848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1586173,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.46999999999999997},"min":{"1min":0.38700000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.49299999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":23.486999999999998,"5min":5.2149999999999999,"15min":2.1699999999999999},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":462.93700000000001,"5min":462.93700000000001,"15min":462.93700000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.94399999999999995,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84399999999999997,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89800000000000002,"5min":1.97,"15min":1.97},"last":0.93100000000000005}]},{"osd":8,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.0720000000000001,"15min":1.1519999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":10.916,"15min":10.916},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.623,"5min":1.218,"15min":1.3169999999999999},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":13.147,"15min":13.147},"last":0.59499999999999997}]},{"osd":10,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":5.8819999999999997,"5min":1.702,"15min":1.006},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":127.033,"5min":127.033,"15min":127.033},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.90600000000000003,"15min":0.95299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92900000000000005,"5min":6.4539999999999997,"15min":6.4539999999999997},"last":0.877}]},{"osd":11,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":10.221,"5min":2.5840000000000001,"15min":1.3109999999999999},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":229.28899999999999,"5min":229.28899999999999,"15min":229.28899999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.88700000000000001,"15min":0.93100000000000005},"min":{"1min":0.42199999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.874,"5min":6.9489999999999998,"15min":6.9489999999999998},"last":0.68600000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":23.875,"15min":27.75},"min":{"1min":0.47899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97699999999999998,"5min":495.95800000000003,"15min":495.95800000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":43.860999999999997,"15min":51.058999999999997},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":914.18499999999995,"15min":914.18499999999995},"last":0.94999999999999996}]},{"osd":13,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":2.7599999999999998,"5min":1.595,"15min":1.401},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":49.841000000000001,"5min":49.841000000000001,"15min":49.841000000000001},"last":0.84699999999999998}]},{"osd":14,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86099999999999999,"5min":1.96,"15min":1.96},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.96199999999999997}]},{"osd":15,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98799999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91000000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.66300000000000003}]}]},{"osd":3,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798228,"kb_used_data":92881448,"kb_used_omap":2127,"kb_used_meta":0,"kb_avail":92881388,"statfs":{"total":95927926784,"available":95110541312,"internally_reserved":61440,"allocated":95110602752,"data_stored":95110602752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2179043,"internal_metadata":0},"hb_peers":[0,2,4,6,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.365,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.91700000000000004}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.57699999999999996,"5min":0.312,"15min":0.312},"max":{"1min":0.99099999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.503}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.81399999999999995,"15min":0.84699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1850000000000001,"5min":7.4720000000000004,"15min":7.4720000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":7.5629999999999997,"15min":8.7230000000000008},"min":{"1min":0.41099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.1100000000000001,"5min":182.87,"15min":182.87},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.5,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":3.202},{"interface":"front","average":{"1min":28.616,"5min":6.2119999999999997,"15min":2.4780000000000002},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":501.65800000000002,"5min":501.65800000000002,"15min":501.65800000000002},"last":0.56200000000000006}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":24.288,"5min":24.288,"15min":24.288},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":500.678,"5min":500.678,"15min":500.678},"last":0.55100000000000005},{"interface":"front","average":{"1min":35.298999999999999,"5min":35.298999999999999,"15min":35.298999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":500.79899999999998,"5min":500.79899999999998,"15min":500.79899999999998},"last":0.52200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":11.316000000000001,"5min":11.316000000000001,"15min":11.316000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":200.40799999999999,"5min":200.40799999999999,"15min":200.40799999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.59799999999999998}]},{"osd":11,"last update":"Sun Sep 15 01:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.433}]},{"osd":12,"last update":"Sun Sep 15 01:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.61199999999999999}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.78400000000000003,"15min":0.76800000000000002},"min":{"1min":0.67900000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.427,"5min":1.637,"15min":1.637},"last":0.624},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0169999999999999,"15min":1.0389999999999999},"min":{"1min":0.63200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.476,"5min":7.4500000000000002,"15min":7.4500000000000002},"last":0.57999999999999996}]},{"osd":14,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53200000000000003}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.52200000000000002}]}]},{"osd":0,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798352,"kb_used_data":92881324,"kb_used_omap":5384,"kb_used_meta":0,"kb_avail":92881264,"statfs":{"total":95927926784,"available":95110414336,"internally_reserved":61440,"allocated":95110475776,"data_stored":95110475776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5513250,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.872},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.113,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.113}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":31.643999999999998,"15min":36.808999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.488,"5min":814.68299999999999,"15min":814.68299999999999},"last":0.624},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.63700000000000001,"15min":0.65200000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1890000000000001,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":40.415999999999997,"5min":8.5380000000000003,"15min":3.2250000000000001},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":799.04999999999995,"5min":799.04999999999995,"15min":799.04999999999995},"last":0.66500000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69399999999999995,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.2769999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.264,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.133}]},{"osd":7,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":46.667999999999999,"5min":9.8330000000000002,"15min":3.694},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":920.58399999999995,"5min":920.58399999999995,"15min":920.58399999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.41599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0629999999999999,"5min":1.304,"15min":1.304},"last":0.84799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.222,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.222}]},{"osd":10,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.78500000000000003}]},{"osd":12,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":1.026},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.58899999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":25.651,"5min":25.651,"15min":25.651},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":501.07400000000001,"5min":501.07400000000001,"15min":501.07400000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":44.5,"5min":44.5,"15min":44.5},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":501.08800000000002,"5min":501.08800000000002,"15min":501.08800000000002},"last":1.2070000000000001}]},{"osd":15,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.403,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.96999999999999997}]}]},{"osd":13,"up_from":21,"seq":90194313245,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799440,"kb_used_data":92880240,"kb_used_omap":4484,"kb_used_meta":0,"kb_avail":92880176,"statfs":{"total":95927926784,"available":95109300224,"internally_reserved":65536,"allocated":95109365760,"data_stored":95109365760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4592085,"internal_metadata":0},"hb_peers":[2,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.1129999999999995,"5min":8.1129999999999995,"15min":8.1129999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":8.1430000000000007,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":0.60899999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":19.710000000000001,"5min":19.710000000000001,"15min":19.710000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":342.56099999999998,"5min":342.56099999999998,"15min":342.56099999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":28.227,"5min":28.227,"15min":28.227},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":495.40800000000002,"5min":495.40800000000002,"15min":495.40800000000002},"last":0.60099999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.7960000000000003,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.7619999999999996,"5min":5.7619999999999996,"15min":5.7619999999999996},"last":0.64600000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.373}]},{"osd":6,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":18.945,"5min":18.945,"15min":18.945},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":293.34800000000001,"5min":293.34800000000001,"15min":293.34800000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.40100000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.4779999999999998,"5min":6.4779999999999998,"15min":6.4779999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":5.726,"5min":5.726,"15min":5.726},"last":0.73199999999999998}]},{"osd":8,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":44.985999999999997,"5min":44.985999999999997,"15min":44.985999999999997},"last":0.52200000000000002}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.63100000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54000000000000004}]},{"osd":14,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.70899999999999996}]},{"osd":15,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.746}]}]},{"osd":2,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798928,"kb_used_data":92880748,"kb_used_omap":4392,"kb_used_meta":0,"kb_avail":92880688,"statfs":{"total":95927926784,"available":95109824512,"internally_reserved":61440,"allocated":95109885952,"data_stored":95109885952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4498429,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95199999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86599999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.38400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":46.487000000000002,"5min":9.6280000000000001,"15min":3.4849999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":829.03399999999999,"5min":829.03399999999999,"15min":829.03399999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.43099999999999999,"15min":0.41099999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.72899999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":39.168999999999997,"5min":13.167999999999999,"15min":8.8339999999999996},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":655.90899999999999,"5min":655.90899999999999,"15min":655.90899999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61299999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0329999999999999,"5min":1.329,"15min":1.329},"last":0.79700000000000004},{"interface":"front","average":{"1min":54.841000000000001,"5min":11.481,"15min":4.2539999999999996},"min":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":976.00800000000004,"5min":976.00800000000004,"15min":976.00800000000004},"last":0.74399999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63}]},{"osd":11,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.68700000000000006,"15min":0.63600000000000001},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":6.4240000000000004,"5min":6.4240000000000004,"15min":6.4240000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.74399999999999999,"15min":0.68999999999999995},"min":{"1min":0.47299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":6.4390000000000001,"5min":6.4390000000000001,"15min":6.4390000000000001},"last":0.89800000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":47.031999999999996,"5min":9.9469999999999992,"15min":3.766},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":834.505,"5min":834.505,"15min":834.505},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97199999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.71599999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99199999999999999,"5min":1.363,"15min":1.363},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.88}]},{"osd":14,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.92200000000000004}]}]},{"osd":15,"up_from":21,"seq":90194313245,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798300,"kb_used_data":92881368,"kb_used_omap":3193,"kb_used_meta":0,"kb_avail":92881316,"statfs":{"total":95927926784,"available":95110467584,"internally_reserved":53248,"allocated":95110520832,"data_stored":95110520832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3270604,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.79300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.38300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":12.098000000000001,"5min":2.8300000000000001,"15min":1.2849999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":162.16800000000001,"5min":162.16800000000001,"15min":162.16800000000001},"last":0.57099999999999995}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":47.701000000000001,"5min":9.8719999999999999,"15min":3.5670000000000002},"min":{"1min":0.26600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":662.84000000000003,"5min":662.84000000000003,"15min":662.84000000000003},"last":0.35499999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64200000000000002,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57099999999999995,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.40400000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94899999999999995,"5min":1.385,"15min":1.385},"last":0.434},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58599999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":4.798,"15min":5.5069999999999997},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69999999999999996,"5min":90.775999999999996,"15min":90.775999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67100000000000004}]},{"osd":7,"last update":"Sun Sep 15 01:33:36 2024","interfaces":[{"interface":"back","average":{"1min":50.158999999999999,"5min":50.158999999999999,"15min":50.158999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":894.27700000000004,"5min":894.27700000000004,"15min":894.27700000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":49.308,"5min":49.308,"15min":49.308},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":878.12199999999996,"5min":878.12199999999996,"15min":878.12199999999996},"last":0.60399999999999998}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":50.171999999999997,"5min":10.541,"15min":3.9359999999999999},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":645.34900000000005,"5min":645.34900000000005,"15min":645.34900000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":27.992000000000001,"15min":32.540999999999997},"min":{"1min":0.45900000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99099999999999999,"5min":542.06600000000003,"15min":542.06600000000003},"last":0.63500000000000001}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":1.603,"15min":1.603},"last":0.501},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96199999999999997,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.65700000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":1.163,"15min":1.163},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.75,"5min":12.249000000000001,"15min":14.164999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.224,"5min":246.56100000000001,"15min":246.56100000000001},"last":0.70199999999999996}]}]},{"osd":4,"up_from":21,"seq":90194313245,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798196,"kb_used_data":92881444,"kb_used_omap":3058,"kb_used_meta":0,"kb_avail":92881420,"statfs":{"total":95927926784,"available":95110574080,"internally_reserved":24576,"allocated":95110598656,"data_stored":95110598656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3131934,"internal_metadata":0},"hb_peers":[0,3,5,6,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":17.896999999999998,"5min":17.896999999999998,"15min":17.896999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":314.661,"5min":314.661,"15min":314.661},"last":0.42499999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51300000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.747},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":1.127}]},{"osd":3,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.55900000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0940000000000001,"5min":1.179,"15min":1.179},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.45000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.622},{"interface":"front","average":{"1min":42.378,"5min":24.170000000000002,"15min":21.135000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":918.75699999999995,"5min":918.75699999999995,"15min":918.75699999999995},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":7.5110000000000001,"5min":2.1309999999999998,"15min":1.2350000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":142.26599999999999,"5min":142.26599999999999,"15min":142.26599999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.57899999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1719999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0840000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.871}]},{"osd":9,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78300000000000003,"15min":0.77900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.59499999999999997}]},{"osd":10,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":40.198999999999998,"5min":8.7089999999999996,"15min":3.4609999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":828.54499999999996,"5min":828.54499999999996,"15min":828.54499999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":14.634,"5min":3.5920000000000001,"15min":1.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":291.08800000000002,"5min":291.08800000000002,"15min":291.08800000000002},"last":0.67800000000000005}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":28.721,"5min":28.721,"15min":28.721},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":558.56700000000001,"5min":558.56700000000001,"15min":558.56700000000001},"last":0.57999999999999996}]},{"osd":12,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1799999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.5}]},{"osd":13,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.71699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.61099999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3169999999999999,"5min":1.325,"15min":1.325},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.90500000000000003,"15min":0.90700000000000003},"min":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.256,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.64300000000000002}]}]},{"osd":5,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798296,"kb_used_data":92881368,"kb_used_omap":3055,"kb_used_meta":0,"kb_avail":92881320,"statfs":{"total":95927926784,"available":95110471680,"internally_reserved":49152,"allocated":95110520832,"data_stored":95110520832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3128858,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.499},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.70799999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.88300000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72599999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.628,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.48,"5min":1.524,"15min":1.524},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.70199999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.80800000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92200000000000004,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.60599999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":36.902999999999999,"5min":7.8339999999999996,"15min":2.9889999999999999},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":764.33399999999995,"5min":764.33399999999995,"15min":764.33399999999995},"last":0.441},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.39200000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1899999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":17.335999999999999,"5min":22.786000000000001,"15min":23.693999999999999},"min":{"1min":0.496,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":346.49200000000002,"5min":492.697,"15min":492.697},"last":0.78800000000000003}]},{"osd":15,"last update":"Sun Sep 15 01:33:26 2024","interfaces":[{"interface":"back","average":{"1min":13.939,"5min":13.939,"15min":13.939},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":291.12599999999998,"5min":291.12599999999998,"15min":291.12599999999998},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.93700000000000006}]}]},{"osd":6,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798300,"kb_used_data":92881364,"kb_used_omap":3133,"kb_used_meta":0,"kb_avail":92881316,"statfs":{"total":95927926784,"available":95110467584,"internally_reserved":49152,"allocated":95110516736,"data_stored":95110516736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3208697,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.85699999999999998,"15min":0.88100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.165,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77600000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.878,"5min":1.264,"15min":1.264},"last":0.60499999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.53800000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77800000000000002,"5min":1.163,"15min":1.163},"last":0.755}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.379,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83799999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.746,"15min":0.746},"last":0.49399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.90200000000000002}]},{"osd":10,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":61.164000000000001,"5min":12.723000000000001,"15min":4.649},"min":{"1min":0.48099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":656.21000000000004,"5min":656.21000000000004,"15min":656.21000000000004},"last":0.83399999999999996}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":11.742000000000001,"5min":2.8580000000000001,"15min":1.3779999999999999},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":200.52799999999999,"5min":200.52799999999999,"15min":200.52799999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.64600000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82999999999999996,"5min":1.353,"15min":1.353},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.99199999999999999,"15min":1.036},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97999999999999998,"5min":5.8470000000000004,"15min":5.8470000000000004},"last":0.85799999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.55800000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.55200000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.034,"5min":1.212,"15min":1.212},"last":0.91500000000000004}]}]},{"osd":7,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881452,"kb_used_omap":2577,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":61440,"allocated":95110606848,"data_stored":95110606848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2639163,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.48199999999999998,"15min":0.47099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":47.609999999999999,"15min":55.453000000000003},"min":{"1min":0.36699999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.69999999999999996,"5min":874.39499999999998,"15min":874.39499999999998},"last":0.38300000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":4.7350000000000003,"5min":4.7350000000000003,"15min":4.7350000000000003},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.33700000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.58299999999999996}]},{"osd":6,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":1.165,"15min":1.165},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":1.0880000000000001,"15min":1.167},"min":{"1min":0.42799999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":5.6539999999999999,"15min":5.6539999999999999},"last":0.52400000000000002}]},{"osd":8,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55900000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":4.4359999999999999,"15min":5.069},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84899999999999998,"5min":62.991999999999997,"15min":62.991999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.50700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67600000000000005,"15min":0.68400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":1.21,"15min":1.21},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":43.418999999999997,"15min":50.549999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93400000000000005,"5min":792.73599999999999,"15min":792.73599999999999},"last":0.47299999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.091,"15min":1.157},"min":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92100000000000004,"5min":8.1489999999999991,"15min":8.1489999999999991},"last":0.68999999999999995}]},{"osd":14,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.47199999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.878,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":41.905000000000001,"15min":48.776000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.83099999999999996,"5min":773.18799999999999,"15min":773.18799999999999},"last":0.72699999999999998}]},{"osd":15,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.1220000000000001,"15min":1.216},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79000000000000004,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83399999999999996,"5min":1.018,"15min":1.018},"last":0.60599999999999998}]}]},{"osd":8,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798352,"kb_used_data":92881324,"kb_used_omap":3898,"kb_used_meta":0,"kb_avail":92881264,"statfs":{"total":95927926784,"available":95110414336,"internally_reserved":61440,"allocated":95110475776,"data_stored":95110475776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3991961,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.373},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.626,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.40500000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.503},{"interface":"front","average":{"1min":24.794,"5min":24.794,"15min":24.794},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":413.66800000000001,"5min":413.66800000000001,"15min":413.66800000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55300000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.44800000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72499999999999998,"5min":1.093,"15min":1.093},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.44700000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.95599999999999996,"5min":1.073,"15min":1.073},"last":0.57499999999999996}]},{"osd":7,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":30.347000000000001,"5min":6.5869999999999997,"15min":2.6269999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":505.29300000000001,"5min":505.29300000000001,"15min":505.29300000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":12.156000000000001,"5min":2.9510000000000001,"15min":1.417},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":196.00299999999999,"5min":196.00299999999999,"15min":196.00299999999999},"last":0.79900000000000004}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.46899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97699999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.75600000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0249999999999999,"5min":1.208,"15min":1.208},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.004,"5min":1.351,"15min":1.351},"last":0.84799999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.503,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66100000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":1.054,"15min":1.054},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98999999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.67500000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798172,"kb_used_data":92881492,"kb_used_omap":3280,"kb_used_meta":0,"kb_avail":92881444,"statfs":{"total":95927926784,"available":95110598656,"internally_reserved":49152,"allocated":95110647808,"data_stored":95110647808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3358734,"internal_metadata":0},"hb_peers":[3,5,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.899},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75700000000000001,"15min":0.752},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":3.9750000000000001,"5min":3.9750000000000001,"15min":3.9750000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":3.9039999999999999,"5min":4.3150000000000004,"15min":4.383},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":56.188000000000002,"5min":64.938999999999993,"15min":64.938999999999993},"last":0.84799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":4.7960000000000003,"5min":1.5009999999999999,"15min":0.95199999999999996},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":75.778999999999996,"5min":75.778999999999996,"15min":75.778999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.75900000000000001,"15min":0.78100000000000003},"min":{"1min":0.45300000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84599999999999997,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.47999999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":44.508000000000003,"15min":51.82},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96299999999999997,"5min":500.92099999999999,"15min":500.92099999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.41399999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.83699999999999997}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.52500000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84199999999999997,"15min":0.87},"min":{"1min":0.503,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0229999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.82299999999999995}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.4460000000000002},{"interface":"front","average":{"1min":36.435000000000002,"5min":36.435000000000002,"15min":36.435000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":641.649,"5min":641.649,"15min":641.649},"last":2.4740000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.048},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.92800000000000005}]},{"osd":15,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.93899999999999995,"15min":0.97299999999999998},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.036,"5min":6.0670000000000002,"15min":6.0670000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0089999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.94799999999999995}]}]},{"osd":11,"up_from":21,"seq":90194313244,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798104,"kb_used_data":92881548,"kb_used_omap":3333,"kb_used_meta":0,"kb_avail":92881512,"statfs":{"total":95927926784,"available":95110668288,"internally_reserved":36864,"allocated":95110705152,"data_stored":95110705152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3413852,"internal_metadata":0},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":46.469000000000001,"5min":46.469000000000001,"15min":46.469000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":873.95500000000004,"5min":873.95500000000004,"15min":873.95500000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.954,"5min":1.954,"15min":1.954},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":28.431999999999999,"5min":28.431999999999999,"15min":28.431999999999999},"last":0.50900000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.443}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":13.093,"5min":3.125,"15min":1.4630000000000001},"min":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":250.767,"5min":250.767,"15min":250.767},"last":0.625},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84899999999999998,"5min":0.871,"15min":0.871},"last":0.84899999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70999999999999996}]},{"osd":6,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.73799999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.60499999999999998}]},{"osd":9,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.747}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86899999999999999,"5min":1.042,"15min":1.042},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90200000000000002,"5min":2.262,"15min":2.262},"last":0.90200000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.79000000000000004,"15min":0.81799999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93400000000000005,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.82099999999999995,"15min":0.84699999999999998},"min":{"1min":0.45300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85999999999999999,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":0.72599999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":1.903,"5min":0.95599999999999996,"15min":0.79800000000000004},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":25.056999999999999,"5min":25.056999999999999,"15min":25.056999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.442,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.91300000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":4.4009999999999998,"5min":13.781000000000001,"15min":15.343999999999999},"min":{"1min":0.51200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":74.078000000000003,"5min":310.78199999999998,"15min":310.78199999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.436,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97099999999999997,"5min":2.238,"15min":2.238},"last":0.97099999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":12,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798284,"kb_used_data":92881380,"kb_used_omap":1469,"kb_used_meta":0,"kb_avail":92881332,"statfs":{"total":95927926784,"available":95110483968,"internally_reserved":49152,"allocated":95110533120,"data_stored":95110533120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1505179,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.28000000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":8.0050000000000008,"5min":8.0050000000000008,"15min":8.0050000000000008},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":0.35599999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.47099999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.51500000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":24.937000000000001,"5min":24.937000000000001,"15min":24.937000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":488.69,"5min":488.69,"15min":488.69},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5419999999999998,"5min":3.5419999999999998,"15min":3.5419999999999998},"last":0.29599999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.6309999999999998,"5min":3.6309999999999998,"15min":3.6309999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.57899999999999996}]},{"osd":11,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.499}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.433},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.40400000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.54300000000000004}]},{"osd":15,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.56899999999999995}]}]},{"osd":1,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798292,"kb_used_data":92881396,"kb_used_omap":3216,"kb_used_meta":0,"kb_avail":92881324,"statfs":{"total":95927926784,"available":95110475776,"internally_reserved":73728,"allocated":95110549504,"data_stored":95110549504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3294094,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":55.167999999999999,"5min":11.429,"15min":4.1390000000000002},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.31},"max":{"1min":875.06100000000004,"5min":875.06100000000004,"15min":875.06100000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.754,"15min":0.754},"last":0.504}]},{"osd":2,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79100000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61299999999999999,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91100000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59099999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.79300000000000004}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.29999999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79200000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":28.789000000000001,"15min":33.491},"min":{"1min":0.38400000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82999999999999996,"5min":635.46400000000006,"15min":635.46400000000006},"last":0.41999999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.53800000000000003},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71199999999999997,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.80500000000000005,"15min":0.85699999999999998},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72699999999999998,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.69799999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":54.337000000000003,"5min":11.387,"15min":4.2290000000000001},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":806.31200000000001,"5min":806.31200000000001,"15min":806.31200000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68000000000000005,"15min":0.68600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83799999999999997,"5min":1.341,"15min":1.341},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.75,"15min":0.751},"min":{"1min":0.52600000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.4379999999999999,"5min":1.825,"15min":1.825},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.48499999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0469999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.82499999999999996}]},{"osd":8,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":4.0830000000000002,"5min":1.395,"15min":0.94699999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":55.694000000000003,"5min":55.694000000000003,"15min":55.694000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69199999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.88900000000000001,"5min":1.492,"15min":1.492},"last":0.65500000000000003}]},{"osd":10,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":2.149,"15min":2.222},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":17.527000000000001,"5min":27.920000000000002,"15min":27.920000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":8.1069999999999993,"15min":9.3320000000000007},"min":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.026,"5min":166.16200000000001,"15min":166.16200000000001},"last":0.80800000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.755},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.496,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84999999999999998,"5min":1.393,"15min":1.393},"last":0.60999999999999999}]}]}],"pool_statfs":[]}} 2024-09-15T01:34:25.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:34:25.321 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:34:25.322 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:34:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-15T01:34:25.095499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":5,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":12773700,"kb_used_data":1486100956,"kb_used_omap":52112,"kb_used_meta":0,"kb_avail":1486100156,"statfs":{"total":1534846828544,"available":1521766559744,"internally_reserved":819200,"allocated":1521767378944,"data_stored":1521767378944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53363571,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001207"},"pg_stats":[{"pgid":"1.0","version":"106'114","reported_seq":266,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888285+0000","last_change":"2024-09-15T01:33:49.902092+0000","last_active":"2024-09-15T01:34:19.888285+0000","last_peered":"2024-09-15T01:34:19.888285+0000","last_clean":"2024-09-15T01:34:19.888285+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:34:19.888285+0000","last_undegraded":"2024-09-15T01:34:19.888285+0000","last_fullsized":"2024-09-15T01:34:19.888285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'114","last_scrub_stamp":"2024-09-15T01:33:49.902036+0000","last_deep_scrub":"106'114","last_deep_scrub_stamp":"2024-09-15T01:33:49.902036+0000","last_clean_scrub_stamp":"2024-09-15T01:33:49.902036+0000","objects_scrubbed":2,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:12.193331+0000","scrub_duration":0.010874557,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"105'2","reported_seq":128,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.341551+0000","last_change":"2024-09-15T01:33:48.491848+0000","last_active":"2024-09-15T01:34:19.341551+0000","last_peered":"2024-09-15T01:34:19.341551+0000","last_clean":"2024-09-15T01:34:19.341551+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:34:19.341551+0000","last_undegraded":"2024-09-15T01:34:19.341551+0000","last_fullsized":"2024-09-15T01:34:19.341551+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'2","last_scrub_stamp":"2024-09-15T01:33:48.491771+0000","last_deep_scrub":"105'2","last_deep_scrub_stamp":"2024-09-15T01:33:48.491771+0000","last_clean_scrub_stamp":"2024-09-15T01:33:48.491771+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:01.566416+0000","scrub_duration":0.0025105769999999999,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.794589+0000","last_change":"2024-09-15T01:33:46.379860+0000","last_active":"2024-09-15T01:34:20.794589+0000","last_peered":"2024-09-15T01:34:20.794589+0000","last_clean":"2024-09-15T01:34:20.794589+0000","last_became_active":"2024-09-15T01:33:23.083242+0000","last_became_peered":"2024-09-15T01:33:23.083242+0000","last_unstale":"2024-09-15T01:34:20.794589+0000","last_undegraded":"2024-09-15T01:34:20.794589+0000","last_fullsized":"2024-09-15T01:34:20.794589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:46.379809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:46.379809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:49.087667+0000","scrub_duration":0.001946382,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8],"acting":[10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888248+0000","last_change":"2024-09-15T01:33:50.941845+0000","last_active":"2024-09-15T01:34:19.888248+0000","last_peered":"2024-09-15T01:34:19.888248+0000","last_clean":"2024-09-15T01:34:19.888248+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:34:19.888248+0000","last_undegraded":"2024-09-15T01:34:19.888248+0000","last_fullsized":"2024-09-15T01:34:19.888248+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:50.941790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:50.941790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:10.537027+0000","scrub_duration":0.0017165430000000001,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":133,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.484672+0000","last_change":"2024-09-15T01:33:46.870657+0000","last_active":"2024-09-15T01:34:19.484672+0000","last_peered":"2024-09-15T01:34:19.484672+0000","last_clean":"2024-09-15T01:34:19.484672+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:34:19.484672+0000","last_undegraded":"2024-09-15T01:34:19.484672+0000","last_fullsized":"2024-09-15T01:34:19.484672+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-15T01:33:46.870584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:46.870584+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:14.306792+0000","scrub_duration":0.0027185820000000002,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888035+0000","last_change":"2024-09-15T01:33:41.548849+0000","last_active":"2024-09-15T01:34:19.888035+0000","last_peered":"2024-09-15T01:34:19.888035+0000","last_clean":"2024-09-15T01:34:19.888035+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:34:19.888035+0000","last_undegraded":"2024-09-15T01:34:19.888035+0000","last_fullsized":"2024-09-15T01:34:19.888035+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:41.548799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:41.548799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:55.947522+0000","scrub_duration":0.001258681,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888166+0000","last_change":"2024-09-15T01:33:41.543896+0000","last_active":"2024-09-15T01:34:19.888166+0000","last_peered":"2024-09-15T01:34:19.888166+0000","last_clean":"2024-09-15T01:34:19.888166+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:34:19.888166+0000","last_undegraded":"2024-09-15T01:34:19.888166+0000","last_fullsized":"2024-09-15T01:34:19.888166+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:41.543850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:41.543850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:56.945346+0000","scrub_duration":0.0015955419999999999,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.587067+0000","last_change":"2024-09-15T01:33:39.638452+0000","last_active":"2024-09-15T01:34:20.587067+0000","last_peered":"2024-09-15T01:34:20.587067+0000","last_clean":"2024-09-15T01:34:20.587067+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:34:20.587067+0000","last_undegraded":"2024-09-15T01:34:20.587067+0000","last_fullsized":"2024-09-15T01:34:20.587067+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:39.638404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:39.638404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:59.511423+0000","scrub_duration":0.001362835,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":129,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.794259+0000","last_change":"2024-09-15T01:33:38.175676+0000","last_active":"2024-09-15T01:34:20.794259+0000","last_peered":"2024-09-15T01:34:20.794259+0000","last_clean":"2024-09-15T01:34:20.794259+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:34:20.794259+0000","last_undegraded":"2024-09-15T01:34:20.794259+0000","last_fullsized":"2024-09-15T01:34:20.794259+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:38.175625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:38.175625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:44.144155+0000","scrub_duration":0.001339526,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881456,"kb_used_omap":3954,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":65536,"allocated":95110610944,"data_stored":95110610944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4049435,"internal_metadata":0},"hb_peers":[1,2,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":16.196999999999999,"5min":27.753,"15min":29.678999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":299.721,"5min":574.26900000000001,"15min":574.26900000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":18.404,"5min":4.0270000000000001,"15min":1.631},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":342.08300000000003,"5min":342.08300000000003,"15min":342.08300000000003},"last":0.35799999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:33:20 2024","interfaces":[{"interface":"back","average":{"1min":32.810000000000002,"5min":32.810000000000002,"15min":32.810000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":644.97699999999998,"5min":644.97699999999998,"15min":644.97699999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":805.39499999999998}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.71399999999999997,"15min":0.73999999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.69299999999999995,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":11.887,"5min":2.8730000000000002,"15min":1.371},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":214.994,"5min":214.994,"15min":214.994},"last":0.66900000000000004}]},{"osd":5,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47699999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.502}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82899999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.621},{"interface":"front","average":{"1min":10.272,"5min":2.625,"15min":1.351},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":173.11099999999999,"5min":173.11099999999999,"15min":173.11099999999999},"last":0.48599999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":6.056,"15min":6.9420000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.589,"5min":134.12200000000001,"15min":134.12200000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":45.521000000000001,"5min":9.7249999999999996,"15min":3.7589999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":846.97400000000005,"5min":846.97400000000005,"15min":846.97400000000005},"last":0.65400000000000003}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.56599999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.5649999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":17.137,"5min":4.0940000000000003,"15min":1.9199999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":313.495,"5min":313.495,"15min":313.495},"last":0.72499999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.69399999999999995}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81399999999999995,"15min":0.81299999999999994},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.70899999999999996}]}]},{"osd":9,"up_from":21,"seq":90194313244,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881452,"kb_used_omap":1548,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":61440,"allocated":95110606848,"data_stored":95110606848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1586173,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.46999999999999997},"min":{"1min":0.38700000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.49299999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":23.486999999999998,"5min":5.2149999999999999,"15min":2.1699999999999999},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":462.93700000000001,"5min":462.93700000000001,"15min":462.93700000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.94399999999999995,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84399999999999997,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89800000000000002,"5min":1.97,"15min":1.97},"last":0.93100000000000005}]},{"osd":8,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.0720000000000001,"15min":1.1519999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":10.916,"15min":10.916},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.623,"5min":1.218,"15min":1.3169999999999999},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":13.147,"15min":13.147},"last":0.59499999999999997}]},{"osd":10,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":5.8819999999999997,"5min":1.702,"15min":1.006},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":127.033,"5min":127.033,"15min":127.033},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.90600000000000003,"15min":0.95299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92900000000000005,"5min":6.4539999999999997,"15min":6.4539999999999997},"last":0.877}]},{"osd":11,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":10.221,"5min":2.5840000000000001,"15min":1.3109999999999999},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":229.28899999999999,"5min":229.28899999999999,"15min":229.28899999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.88700000000000001,"15min":0.93100000000000005},"min":{"1min":0.42199999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.874,"5min":6.9489999999999998,"15min":6.9489999999999998},"last":0.68600000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":23.875,"15min":27.75},"min":{"1min":0.47899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97699999999999998,"5min":495.95800000000003,"15min":495.95800000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":43.860999999999997,"15min":51.058999999999997},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":914.18499999999995,"15min":914.18499999999995},"last":0.94999999999999996}]},{"osd":13,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":2.7599999999999998,"5min":1.595,"15min":1.401},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":49.841000000000001,"5min":49.841000000000001,"15min":49.841000000000001},"last":0.84699999999999998}]},{"osd":14,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86099999999999999,"5min":1.96,"15min":1.96},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.96199999999999997}]},{"osd":15,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98799999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91000000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.66300000000000003}]}]},{"osd":3,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798228,"kb_used_data":92881448,"kb_used_omap":2127,"kb_used_meta":0,"kb_avail":92881388,"statfs":{"total":95927926784,"available":95110541312,"internally_reserved":61440,"allocated":95110602752,"data_stored":95110602752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2179043,"internal_metadata":0},"hb_peers":[0,2,4,6,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.365,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.91700000000000004}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.57699999999999996,"5min":0.312,"15min":0.312},"max":{"1min":0.99099999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.503}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.81399999999999995,"15min":0.84699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1850000000000001,"5min":7.4720000000000004,"15min":7.4720000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":7.5629999999999997,"15min":8.7230000000000008},"min":{"1min":0.41099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.1100000000000001,"5min":182.87,"15min":182.87},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.5,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":3.202},{"interface":"front","average":{"1min":28.616,"5min":6.2119999999999997,"15min":2.4780000000000002},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":501.65800000000002,"5min":501.65800000000002,"15min":501.65800000000002},"last":0.56200000000000006}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":24.288,"5min":24.288,"15min":24.288},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":500.678,"5min":500.678,"15min":500.678},"last":0.55100000000000005},{"interface":"front","average":{"1min":35.298999999999999,"5min":35.298999999999999,"15min":35.298999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":500.79899999999998,"5min":500.79899999999998,"15min":500.79899999999998},"last":0.52200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":11.316000000000001,"5min":11.316000000000001,"15min":11.316000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":200.40799999999999,"5min":200.40799999999999,"15min":200.40799999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.59799999999999998}]},{"osd":11,"last update":"Sun Sep 15 01:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.433}]},{"osd":12,"last update":"Sun Sep 15 01:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.61199999999999999}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.78400000000000003,"15min":0.76800000000000002},"min":{"1min":0.67900000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.427,"5min":1.637,"15min":1.637},"last":0.624},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0169999999999999,"15min":1.0389999999999999},"min":{"1min":0.63200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.476,"5min":7.4500000000000002,"15min":7.4500000000000002},"last":0.57999999999999996}]},{"osd":14,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53200000000000003}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.52200000000000002}]}]},{"osd":0,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798352,"kb_used_data":92881324,"kb_used_omap":5384,"kb_used_meta":0,"kb_avail":92881264,"statfs":{"total":95927926784,"available":95110414336,"internally_reserved":61440,"allocated":95110475776,"data_stored":95110475776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5513250,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.872},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.113,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.113}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":31.643999999999998,"15min":36.808999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.488,"5min":814.68299999999999,"15min":814.68299999999999},"last":0.624},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.63700000000000001,"15min":0.65200000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1890000000000001,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":40.415999999999997,"5min":8.5380000000000003,"15min":3.2250000000000001},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":799.04999999999995,"5min":799.04999999999995,"15min":799.04999999999995},"last":0.66500000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69399999999999995,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.2769999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.264,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.133}]},{"osd":7,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":46.667999999999999,"5min":9.8330000000000002,"15min":3.694},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":920.58399999999995,"5min":920.58399999999995,"15min":920.58399999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.41599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0629999999999999,"5min":1.304,"15min":1.304},"last":0.84799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.222,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.222}]},{"osd":10,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.78500000000000003}]},{"osd":12,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":1.026},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.58899999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":25.651,"5min":25.651,"15min":25.651},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":501.07400000000001,"5min":501.07400000000001,"15min":501.07400000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":44.5,"5min":44.5,"15min":44.5},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":501.08800000000002,"5min":501.08800000000002,"15min":501.08800000000002},"last":1.2070000000000001}]},{"osd":15,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.403,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.96999999999999997}]}]},{"osd":13,"up_from":21,"seq":90194313245,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799440,"kb_used_data":92880240,"kb_used_omap":4484,"kb_used_meta":0,"kb_avail":92880176,"statfs":{"total":95927926784,"available":95109300224,"internally_reserved":65536,"allocated":95109365760,"data_stored":95109365760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4592085,"internal_metadata":0},"hb_peers":[2,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.1129999999999995,"5min":8.1129999999999995,"15min":8.1129999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":8.1430000000000007,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":0.60899999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":19.710000000000001,"5min":19.710000000000001,"15min":19.710000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":342.56099999999998,"5min":342.56099999999998,"15min":342.56099999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":28.227,"5min":28.227,"15min":28.227},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":495.40800000000002,"5min":495.40800000000002,"15min":495.40800000000002},"last":0.60099999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.7960000000000003,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.7619999999999996,"5min":5.7619999999999996,"15min":5.7619999999999996},"last":0.64600000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.373}]},{"osd":6,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":18.945,"5min":18.945,"15min":18.945},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":293.34800000000001,"5min":293.34800000000001,"15min":293.34800000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.40100000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.4779999999999998,"5min":6.4779999999999998,"15min":6.4779999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":5.726,"5min":5.726,"15min":5.726},"last":0.73199999999999998}]},{"osd":8,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":44.985999999999997,"5min":44.985999999999997,"15min":44.985999999999997},"last":0.52200000000000002}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.63100000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54000000000000004}]},{"osd":14,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.70899999999999996}]},{"osd":15,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.746}]}]},{"osd":2,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798928,"kb_used_data":92880748,"kb_used_omap":4392,"kb_used_meta":0,"kb_avail":92880688,"statfs":{"total":95927926784,"available":95109824512,"internally_reserved":61440,"allocated":95109885952,"data_stored":95109885952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4498429,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95199999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86599999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.38400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":46.487000000000002,"5min":9.6280000000000001,"15min":3.4849999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":829.03399999999999,"5min":829.03399999999999,"15min":829.03399999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.43099999999999999,"15min":0.41099999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.72899999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":39.168999999999997,"5min":13.167999999999999,"15min":8.8339999999999996},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":655.90899999999999,"5min":655.90899999999999,"15min":655.90899999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61299999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0329999999999999,"5min":1.329,"15min":1.329},"last":0.79700000000000004},{"interface":"front","average":{"1min":54.841000000000001,"5min":11.481,"15min":4.2539999999999996},"min":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":976.00800000000004,"5min":976.00800000000004,"15min":976.00800000000004},"last":0.74399999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63}]},{"osd":11,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.68700000000000006,"15min":0.63600000000000001},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":6.4240000000000004,"5min":6.4240000000000004,"15min":6.4240000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.74399999999999999,"15min":0.68999999999999995},"min":{"1min":0.47299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":6.4390000000000001,"5min":6.4390000000000001,"15min":6.4390000000000001},"last":0.89800000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":47.031999999999996,"5min":9.9469999999999992,"15min":3.766},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":834.505,"5min":834.505,"15min":834.505},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97199999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.71599999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99199999999999999,"5min":1.363,"15min":1.363},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.88}]},{"osd":14,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.92200000000000004}]}]},{"osd":15,"up_from":21,"seq":90194313246,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798156,"kb_used_data":92881472,"kb_used_omap":3194,"kb_used_meta":0,"kb_avail":92881460,"statfs":{"total":95927926784,"available":95110615040,"internally_reserved":12288,"allocated":95110627328,"data_stored":95110627328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3271314,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.79300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":12.098000000000001,"5min":2.8300000000000001,"15min":1.2849999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":162.16800000000001,"5min":162.16800000000001,"15min":162.16800000000001},"last":0.76500000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":47.701000000000001,"5min":9.8719999999999999,"15min":3.5670000000000002},"min":{"1min":0.26600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":662.84000000000003,"5min":662.84000000000003,"15min":662.84000000000003},"last":0.39700000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64200000000000002,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57099999999999995,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.40400000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94899999999999995,"5min":1.385,"15min":1.385},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.85699999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":4.798,"15min":5.5069999999999997},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69999999999999996,"5min":90.775999999999996,"15min":90.775999999999996},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Sep 15 01:33:36 2024","interfaces":[{"interface":"back","average":{"1min":50.158999999999999,"5min":50.158999999999999,"15min":50.158999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":894.27700000000004,"5min":894.27700000000004,"15min":894.27700000000004},"last":0.497},{"interface":"front","average":{"1min":49.308,"5min":49.308,"15min":49.308},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":878.12199999999996,"5min":878.12199999999996,"15min":878.12199999999996},"last":0.61299999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":50.171999999999997,"5min":10.541,"15min":3.9359999999999999},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":645.34900000000005,"5min":645.34900000000005,"15min":645.34900000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":27.992000000000001,"15min":32.540999999999997},"min":{"1min":0.45900000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99099999999999999,"5min":542.06600000000003,"15min":542.06600000000003},"last":0.59699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":1.603,"15min":1.603},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96199999999999997,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.78200000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":1.163,"15min":1.163},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75,"5min":12.249000000000001,"15min":14.164999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.224,"5min":246.56100000000001,"15min":246.56100000000001},"last":0.81599999999999995}]}]},{"osd":4,"up_from":21,"seq":90194313245,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798196,"kb_used_data":92881444,"kb_used_omap":3058,"kb_used_meta":0,"kb_avail":92881420,"statfs":{"total":95927926784,"available":95110574080,"internally_reserved":24576,"allocated":95110598656,"data_stored":95110598656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3131934,"internal_metadata":0},"hb_peers":[0,3,5,6,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":17.896999999999998,"5min":17.896999999999998,"15min":17.896999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":314.661,"5min":314.661,"15min":314.661},"last":0.42499999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51300000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.747},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":1.127}]},{"osd":3,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.55900000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0940000000000001,"5min":1.179,"15min":1.179},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.45000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.622},{"interface":"front","average":{"1min":42.378,"5min":24.170000000000002,"15min":21.135000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":918.75699999999995,"5min":918.75699999999995,"15min":918.75699999999995},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":7.5110000000000001,"5min":2.1309999999999998,"15min":1.2350000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":142.26599999999999,"5min":142.26599999999999,"15min":142.26599999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.57899999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1719999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0840000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.871}]},{"osd":9,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78300000000000003,"15min":0.77900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.59499999999999997}]},{"osd":10,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":40.198999999999998,"5min":8.7089999999999996,"15min":3.4609999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":828.54499999999996,"5min":828.54499999999996,"15min":828.54499999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":14.634,"5min":3.5920000000000001,"15min":1.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":291.08800000000002,"5min":291.08800000000002,"15min":291.08800000000002},"last":0.67800000000000005}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":28.721,"5min":28.721,"15min":28.721},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":558.56700000000001,"5min":558.56700000000001,"15min":558.56700000000001},"last":0.57999999999999996}]},{"osd":12,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1799999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.5}]},{"osd":13,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.71699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.61099999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3169999999999999,"5min":1.325,"15min":1.325},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.90500000000000003,"15min":0.90700000000000003},"min":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.256,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.64300000000000002}]}]},{"osd":5,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798296,"kb_used_data":92881368,"kb_used_omap":3055,"kb_used_meta":0,"kb_avail":92881320,"statfs":{"total":95927926784,"available":95110471680,"internally_reserved":49152,"allocated":95110520832,"data_stored":95110520832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3128858,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.499},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.70799999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.88300000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72599999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.628,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.48,"5min":1.524,"15min":1.524},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.70199999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.80800000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92200000000000004,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.60599999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":36.902999999999999,"5min":7.8339999999999996,"15min":2.9889999999999999},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":764.33399999999995,"5min":764.33399999999995,"15min":764.33399999999995},"last":0.441},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.39200000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1899999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":17.335999999999999,"5min":22.786000000000001,"15min":23.693999999999999},"min":{"1min":0.496,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":346.49200000000002,"5min":492.697,"15min":492.697},"last":0.78800000000000003}]},{"osd":15,"last update":"Sun Sep 15 01:33:26 2024","interfaces":[{"interface":"back","average":{"1min":13.939,"5min":13.939,"15min":13.939},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":291.12599999999998,"5min":291.12599999999998,"15min":291.12599999999998},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.93700000000000006}]}]},{"osd":6,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798300,"kb_used_data":92881364,"kb_used_omap":3133,"kb_used_meta":0,"kb_avail":92881316,"statfs":{"total":95927926784,"available":95110467584,"internally_reserved":49152,"allocated":95110516736,"data_stored":95110516736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3208697,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.85699999999999998,"15min":0.88100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.165,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77600000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.878,"5min":1.264,"15min":1.264},"last":0.60499999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.53800000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77800000000000002,"5min":1.163,"15min":1.163},"last":0.755}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.379,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83799999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.746,"15min":0.746},"last":0.49399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.90200000000000002}]},{"osd":10,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":61.164000000000001,"5min":12.723000000000001,"15min":4.649},"min":{"1min":0.48099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":656.21000000000004,"5min":656.21000000000004,"15min":656.21000000000004},"last":0.83399999999999996}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":11.742000000000001,"5min":2.8580000000000001,"15min":1.3779999999999999},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":200.52799999999999,"5min":200.52799999999999,"15min":200.52799999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.64600000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82999999999999996,"5min":1.353,"15min":1.353},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.99199999999999999,"15min":1.036},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97999999999999998,"5min":5.8470000000000004,"15min":5.8470000000000004},"last":0.85799999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.55800000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.55200000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.034,"5min":1.212,"15min":1.212},"last":0.91500000000000004}]}]},{"osd":7,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881452,"kb_used_omap":2577,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":61440,"allocated":95110606848,"data_stored":95110606848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2639163,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.48199999999999998,"15min":0.47099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":47.609999999999999,"15min":55.453000000000003},"min":{"1min":0.36699999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.69999999999999996,"5min":874.39499999999998,"15min":874.39499999999998},"last":0.38300000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":4.7350000000000003,"5min":4.7350000000000003,"15min":4.7350000000000003},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.33700000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.58299999999999996}]},{"osd":6,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":1.165,"15min":1.165},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":1.0880000000000001,"15min":1.167},"min":{"1min":0.42799999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":5.6539999999999999,"15min":5.6539999999999999},"last":0.52400000000000002}]},{"osd":8,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55900000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":4.4359999999999999,"15min":5.069},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84899999999999998,"5min":62.991999999999997,"15min":62.991999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.50700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67600000000000005,"15min":0.68400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":1.21,"15min":1.21},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":43.418999999999997,"15min":50.549999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93400000000000005,"5min":792.73599999999999,"15min":792.73599999999999},"last":0.47299999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.091,"15min":1.157},"min":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92100000000000004,"5min":8.1489999999999991,"15min":8.1489999999999991},"last":0.68999999999999995}]},{"osd":14,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.47199999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.878,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":41.905000000000001,"15min":48.776000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.83099999999999996,"5min":773.18799999999999,"15min":773.18799999999999},"last":0.72699999999999998}]},{"osd":15,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.1220000000000001,"15min":1.216},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79000000000000004,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83399999999999996,"5min":1.018,"15min":1.018},"last":0.60599999999999998}]}]},{"osd":8,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798352,"kb_used_data":92881324,"kb_used_omap":3898,"kb_used_meta":0,"kb_avail":92881264,"statfs":{"total":95927926784,"available":95110414336,"internally_reserved":61440,"allocated":95110475776,"data_stored":95110475776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3991961,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.373},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.626,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.40500000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.503},{"interface":"front","average":{"1min":24.794,"5min":24.794,"15min":24.794},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":413.66800000000001,"5min":413.66800000000001,"15min":413.66800000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55300000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.44800000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72499999999999998,"5min":1.093,"15min":1.093},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.44700000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.95599999999999996,"5min":1.073,"15min":1.073},"last":0.57499999999999996}]},{"osd":7,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":30.347000000000001,"5min":6.5869999999999997,"15min":2.6269999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":505.29300000000001,"5min":505.29300000000001,"15min":505.29300000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":12.156000000000001,"5min":2.9510000000000001,"15min":1.417},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":196.00299999999999,"5min":196.00299999999999,"15min":196.00299999999999},"last":0.79900000000000004}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.46899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97699999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.75600000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0249999999999999,"5min":1.208,"15min":1.208},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.004,"5min":1.351,"15min":1.351},"last":0.84799999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.503,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66100000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":1.054,"15min":1.054},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98999999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.67500000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798172,"kb_used_data":92881492,"kb_used_omap":3280,"kb_used_meta":0,"kb_avail":92881444,"statfs":{"total":95927926784,"available":95110598656,"internally_reserved":49152,"allocated":95110647808,"data_stored":95110647808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3358734,"internal_metadata":0},"hb_peers":[3,5,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.899},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75700000000000001,"15min":0.752},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":3.9750000000000001,"5min":3.9750000000000001,"15min":3.9750000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":3.9039999999999999,"5min":4.3150000000000004,"15min":4.383},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":56.188000000000002,"5min":64.938999999999993,"15min":64.938999999999993},"last":0.84799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":4.7960000000000003,"5min":1.5009999999999999,"15min":0.95199999999999996},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":75.778999999999996,"5min":75.778999999999996,"15min":75.778999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.75900000000000001,"15min":0.78100000000000003},"min":{"1min":0.45300000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84599999999999997,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.47999999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":44.508000000000003,"15min":51.82},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96299999999999997,"5min":500.92099999999999,"15min":500.92099999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.41399999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.83699999999999997}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.52500000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84199999999999997,"15min":0.87},"min":{"1min":0.503,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0229999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.82299999999999995}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.4460000000000002},{"interface":"front","average":{"1min":36.435000000000002,"5min":36.435000000000002,"15min":36.435000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":641.649,"5min":641.649,"15min":641.649},"last":2.4740000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.048},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.92800000000000005}]},{"osd":15,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.93899999999999995,"15min":0.97299999999999998},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.036,"5min":6.0670000000000002,"15min":6.0670000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0089999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.94799999999999995}]}]},{"osd":11,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798032,"kb_used_data":92881596,"kb_used_omap":3335,"kb_used_meta":0,"kb_avail":92881584,"statfs":{"total":95927926784,"available":95110742016,"internally_reserved":12288,"allocated":95110754304,"data_stored":95110754304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3415222,"internal_metadata":0},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":46.469000000000001,"5min":46.469000000000001,"15min":46.469000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":873.95500000000004,"5min":873.95500000000004,"15min":873.95500000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.954,"5min":1.954,"15min":1.954},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":28.431999999999999,"5min":28.431999999999999,"15min":28.431999999999999},"last":0.50900000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":565.90700000000004}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":13.093,"5min":3.125,"15min":1.4630000000000001},"min":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":250.767,"5min":250.767,"15min":250.767},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84899999999999998,"5min":0.871,"15min":0.871},"last":0.495}]},{"osd":5,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.623}]},{"osd":6,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.30199999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.433},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47699999999999998}]},{"osd":9,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51300000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86899999999999999,"5min":1.042,"15min":1.042},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90200000000000002,"5min":2.262,"15min":2.262},"last":0.39500000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.79000000000000004,"15min":0.81799999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93400000000000005,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.82099999999999995,"15min":0.84699999999999998},"min":{"1min":0.45300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85999999999999999,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":0.58599999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":1.903,"5min":0.95599999999999996,"15min":0.79800000000000004},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":25.056999999999999,"5min":25.056999999999999,"15min":25.056999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.442,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.54200000000000004}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":4.4009999999999998,"5min":13.781000000000001,"15min":15.343999999999999},"min":{"1min":0.51200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":74.078000000000003,"5min":310.78199999999998,"15min":310.78199999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.436,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97099999999999997,"5min":2.238,"15min":2.238},"last":0.52400000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":12,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798284,"kb_used_data":92881380,"kb_used_omap":1469,"kb_used_meta":0,"kb_avail":92881332,"statfs":{"total":95927926784,"available":95110483968,"internally_reserved":49152,"allocated":95110533120,"data_stored":95110533120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1505179,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.28000000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":8.0050000000000008,"5min":8.0050000000000008,"15min":8.0050000000000008},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":0.35599999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.47099999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.51500000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":24.937000000000001,"5min":24.937000000000001,"15min":24.937000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":488.69,"5min":488.69,"15min":488.69},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5419999999999998,"5min":3.5419999999999998,"15min":3.5419999999999998},"last":0.29599999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.6309999999999998,"5min":3.6309999999999998,"15min":3.6309999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.57899999999999996}]},{"osd":11,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.499}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.433},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.40400000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.54300000000000004}]},{"osd":15,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.56899999999999995}]}]},{"osd":1,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798292,"kb_used_data":92881396,"kb_used_omap":3216,"kb_used_meta":0,"kb_avail":92881324,"statfs":{"total":95927926784,"available":95110475776,"internally_reserved":73728,"allocated":95110549504,"data_stored":95110549504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3294094,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":55.167999999999999,"5min":11.429,"15min":4.1390000000000002},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.31},"max":{"1min":875.06100000000004,"5min":875.06100000000004,"15min":875.06100000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.754,"15min":0.754},"last":0.504}]},{"osd":2,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79100000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61299999999999999,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91100000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59099999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.79300000000000004}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.29999999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79200000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":28.789000000000001,"15min":33.491},"min":{"1min":0.38400000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82999999999999996,"5min":635.46400000000006,"15min":635.46400000000006},"last":0.41999999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.53800000000000003},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71199999999999997,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.80500000000000005,"15min":0.85699999999999998},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72699999999999998,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.69799999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":54.337000000000003,"5min":11.387,"15min":4.2290000000000001},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":806.31200000000001,"5min":806.31200000000001,"15min":806.31200000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68000000000000005,"15min":0.68600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83799999999999997,"5min":1.341,"15min":1.341},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.75,"15min":0.751},"min":{"1min":0.52600000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.4379999999999999,"5min":1.825,"15min":1.825},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.48499999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0469999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.82499999999999996}]},{"osd":8,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":4.0830000000000002,"5min":1.395,"15min":0.94699999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":55.694000000000003,"5min":55.694000000000003,"15min":55.694000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69199999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.88900000000000001,"5min":1.492,"15min":1.492},"last":0.65500000000000003}]},{"osd":10,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":2.149,"15min":2.222},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":17.527000000000001,"5min":27.920000000000002,"15min":27.920000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":8.1069999999999993,"15min":9.3320000000000007},"min":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.026,"5min":166.16200000000001,"15min":166.16200000000001},"last":0.80800000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.755},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.496,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84999999999999998,"5min":1.393,"15min":1.393},"last":0.60999999999999999}]}]}],"pool_statfs":[]}} 2024-09-15T01:34:25.337 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T01:34:25.337 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-15T01:34:25.632 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T01:34:25.641 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-15T01:34:25.641 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:34:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:34:25.922 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":138,"fsid":"d4803586-ed44-4154-8ee1-6b95ec3e17e7","created":"2024-09-15T01:32:07.285177+0000","modified":"2024-09-15T01:34:21.353291+0000","last_up_change":"2024-09-15T01:32:14.443323+0000","last_in_change":"2024-09-15T01:33:08.068993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:32:15.118634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:32:21.824131+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bec41583-a0a4-4712-a9ec-eec1e88fb60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":28861},{"type":"v1","addr":"172.21.15.42:6809","nonce":28861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":28861},{"type":"v1","addr":"172.21.15.42:6811","nonce":28861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":28861},{"type":"v1","addr":"172.21.15.42:6815","nonce":28861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":28861},{"type":"v1","addr":"172.21.15.42:6813","nonce":28861}]},"public_addr":"172.21.15.42:6809/28861","cluster_addr":"172.21.15.42:6811/28861","heartbeat_back_addr":"172.21.15.42:6815/28861","heartbeat_front_addr":"172.21.15.42:6813/28861","state":["exists","up"]},{"osd":1,"uuid":"2cca66be-8685-4a38-abf8-c864cd5434da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":28508},{"type":"v1","addr":"172.21.15.115:6809","nonce":28508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":28508},{"type":"v1","addr":"172.21.15.115:6811","nonce":28508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6817","nonce":28508},{"type":"v1","addr":"172.21.15.115:6819","nonce":28508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6813","nonce":28508},{"type":"v1","addr":"172.21.15.115:6815","nonce":28508}]},"public_addr":"172.21.15.115:6809/28508","cluster_addr":"172.21.15.115:6811/28508","heartbeat_back_addr":"172.21.15.115:6819/28508","heartbeat_front_addr":"172.21.15.115:6815/28508","state":["exists","up"]},{"osd":2,"uuid":"b8756aac-3a3e-41ba-a6ef-497063cb44df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":28732},{"type":"v1","addr":"172.21.15.152:6809","nonce":28732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":28732},{"type":"v1","addr":"172.21.15.152:6811","nonce":28732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":28732},{"type":"v1","addr":"172.21.15.152:6816","nonce":28732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":28732},{"type":"v1","addr":"172.21.15.152:6813","nonce":28732}]},"public_addr":"172.21.15.152:6809/28732","cluster_addr":"172.21.15.152:6811/28732","heartbeat_back_addr":"172.21.15.152:6816/28732","heartbeat_front_addr":"172.21.15.152:6813/28732","state":["exists","up"]},{"osd":3,"uuid":"554875f5-74c0-4b58-80ee-6ba93748f104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":28835},{"type":"v1","addr":"172.21.15.175:6803","nonce":28835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":28835},{"type":"v1","addr":"172.21.15.175:6805","nonce":28835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":28835},{"type":"v1","addr":"172.21.15.175:6809","nonce":28835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":28835},{"type":"v1","addr":"172.21.15.175:6807","nonce":28835}]},"public_addr":"172.21.15.175:6803/28835","cluster_addr":"172.21.15.175:6805/28835","heartbeat_back_addr":"172.21.15.175:6809/28835","heartbeat_front_addr":"172.21.15.175:6807/28835","state":["exists","up"]},{"osd":4,"uuid":"3fe2c05a-4c22-4dd7-a8c6-282155c82eb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":28860},{"type":"v1","addr":"172.21.15.42:6801","nonce":28860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":28860},{"type":"v1","addr":"172.21.15.42:6803","nonce":28860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":28860},{"type":"v1","addr":"172.21.15.42:6807","nonce":28860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":28860},{"type":"v1","addr":"172.21.15.42:6805","nonce":28860}]},"public_addr":"172.21.15.42:6801/28860","cluster_addr":"172.21.15.42:6803/28860","heartbeat_back_addr":"172.21.15.42:6807/28860","heartbeat_front_addr":"172.21.15.42:6805/28860","state":["exists","up"]},{"osd":5,"uuid":"a301d1ce-3e31-4782-ade3-f9c0f1e6a95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":28509},{"type":"v1","addr":"172.21.15.115:6801","nonce":28509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":28509},{"type":"v1","addr":"172.21.15.115:6803","nonce":28509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":28509},{"type":"v1","addr":"172.21.15.115:6807","nonce":28509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":28509},{"type":"v1","addr":"172.21.15.115:6805","nonce":28509}]},"public_addr":"172.21.15.115:6801/28509","cluster_addr":"172.21.15.115:6803/28509","heartbeat_back_addr":"172.21.15.115:6807/28509","heartbeat_front_addr":"172.21.15.115:6805/28509","state":["exists","up"]},{"osd":6,"uuid":"c7d180be-21c5-4674-a9f2-10a1c7794871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":28731},{"type":"v1","addr":"172.21.15.152:6801","nonce":28731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":28731},{"type":"v1","addr":"172.21.15.152:6803","nonce":28731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":28731},{"type":"v1","addr":"172.21.15.152:6807","nonce":28731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":28731},{"type":"v1","addr":"172.21.15.152:6805","nonce":28731}]},"public_addr":"172.21.15.152:6801/28731","cluster_addr":"172.21.15.152:6803/28731","heartbeat_back_addr":"172.21.15.152:6807/28731","heartbeat_front_addr":"172.21.15.152:6805/28731","state":["exists","up"]},{"osd":7,"uuid":"4d2429ea-772f-4fb9-a421-240f1399f16c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":28836},{"type":"v1","addr":"172.21.15.175:6811","nonce":28836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":28836},{"type":"v1","addr":"172.21.15.175:6813","nonce":28836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":28836},{"type":"v1","addr":"172.21.15.175:6817","nonce":28836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":28836},{"type":"v1","addr":"172.21.15.175:6815","nonce":28836}]},"public_addr":"172.21.15.175:6811/28836","cluster_addr":"172.21.15.175:6813/28836","heartbeat_back_addr":"172.21.15.175:6817/28836","heartbeat_front_addr":"172.21.15.175:6815/28836","state":["exists","up"]},{"osd":8,"uuid":"de7389ab-a29e-4177-9e94-484fc6a94102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":28862},{"type":"v1","addr":"172.21.15.42:6817","nonce":28862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":28862},{"type":"v1","addr":"172.21.15.42:6819","nonce":28862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":28862},{"type":"v1","addr":"172.21.15.42:6823","nonce":28862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":28862},{"type":"v1","addr":"172.21.15.42:6821","nonce":28862}]},"public_addr":"172.21.15.42:6817/28862","cluster_addr":"172.21.15.42:6819/28862","heartbeat_back_addr":"172.21.15.42:6823/28862","heartbeat_front_addr":"172.21.15.42:6821/28862","state":["exists","up"]},{"osd":9,"uuid":"4b4b44da-97d6-41f2-8fc6-57680b10aeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":28510},{"type":"v1","addr":"172.21.15.115:6814","nonce":28510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":28510},{"type":"v1","addr":"172.21.15.115:6818","nonce":28510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":28510},{"type":"v1","addr":"172.21.15.115:6823","nonce":28510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":28510},{"type":"v1","addr":"172.21.15.115:6821","nonce":28510}]},"public_addr":"172.21.15.115:6814/28510","cluster_addr":"172.21.15.115:6818/28510","heartbeat_back_addr":"172.21.15.115:6823/28510","heartbeat_front_addr":"172.21.15.115:6821/28510","state":["exists","up"]},{"osd":10,"uuid":"5f285a07-9057-4575-b911-b3977acd397c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":28733},{"type":"v1","addr":"172.21.15.152:6817","nonce":28733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":28733},{"type":"v1","addr":"172.21.15.152:6819","nonce":28733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":28733},{"type":"v1","addr":"172.21.15.152:6824","nonce":28733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":28733},{"type":"v1","addr":"172.21.15.152:6821","nonce":28733}]},"public_addr":"172.21.15.152:6817/28733","cluster_addr":"172.21.15.152:6819/28733","heartbeat_back_addr":"172.21.15.152:6824/28733","heartbeat_front_addr":"172.21.15.152:6821/28733","state":["exists","up"]},{"osd":11,"uuid":"dacc0f3f-a598-4055-92f4-d41102ceca83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":28837},{"type":"v1","addr":"172.21.15.175:6827","nonce":28837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":28837},{"type":"v1","addr":"172.21.15.175:6829","nonce":28837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":28837},{"type":"v1","addr":"172.21.15.175:6833","nonce":28837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":28837},{"type":"v1","addr":"172.21.15.175:6831","nonce":28837}]},"public_addr":"172.21.15.175:6827/28837","cluster_addr":"172.21.15.175:6829/28837","heartbeat_back_addr":"172.21.15.175:6833/28837","heartbeat_front_addr":"172.21.15.175:6831/28837","state":["exists","up"]},{"osd":12,"uuid":"d218197b-d0db-410c-abff-8b849a21a555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":28866},{"type":"v1","addr":"172.21.15.42:6825","nonce":28866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":28866},{"type":"v1","addr":"172.21.15.42:6827","nonce":28866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":28866},{"type":"v1","addr":"172.21.15.42:6831","nonce":28866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":28866},{"type":"v1","addr":"172.21.15.42:6829","nonce":28866}]},"public_addr":"172.21.15.42:6825/28866","cluster_addr":"172.21.15.42:6827/28866","heartbeat_back_addr":"172.21.15.42:6831/28866","heartbeat_front_addr":"172.21.15.42:6829/28866","state":["exists","up"]},{"osd":13,"uuid":"c9bf5faf-f2ee-482e-b903-28c7718e2063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":28511},{"type":"v1","addr":"172.21.15.115:6825","nonce":28511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":28511},{"type":"v1","addr":"172.21.15.115:6827","nonce":28511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":28511},{"type":"v1","addr":"172.21.15.115:6831","nonce":28511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":28511},{"type":"v1","addr":"172.21.15.115:6829","nonce":28511}]},"public_addr":"172.21.15.115:6825/28511","cluster_addr":"172.21.15.115:6827/28511","heartbeat_back_addr":"172.21.15.115:6831/28511","heartbeat_front_addr":"172.21.15.115:6829/28511","state":["exists","up"]},{"osd":14,"uuid":"cc3f19cf-5a97-440b-a953-7727e6ed6a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6823","nonce":28734},{"type":"v1","addr":"172.21.15.152:6825","nonce":28734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":28734},{"type":"v1","addr":"172.21.15.152:6827","nonce":28734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":28734},{"type":"v1","addr":"172.21.15.152:6831","nonce":28734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":28734},{"type":"v1","addr":"172.21.15.152:6829","nonce":28734}]},"public_addr":"172.21.15.152:6825/28734","cluster_addr":"172.21.15.152:6827/28734","heartbeat_back_addr":"172.21.15.152:6831/28734","heartbeat_front_addr":"172.21.15.152:6829/28734","state":["exists","up"]},{"osd":15,"uuid":"51290c23-e86d-4e05-9ac8-ca53d6a1eb80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":28838},{"type":"v1","addr":"172.21.15.175:6819","nonce":28838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":28838},{"type":"v1","addr":"172.21.15.175:6821","nonce":28838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":28838},{"type":"v1","addr":"172.21.15.175:6825","nonce":28838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":28838},{"type":"v1","addr":"172.21.15.175:6823","nonce":28838}]},"public_addr":"172.21.15.175:6819/28838","cluster_addr":"172.21.15.175:6821/28838","heartbeat_back_addr":"172.21.15.175:6825/28838","heartbeat_front_addr":"172.21.15.175:6823/28838","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:15.503649+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:25.772185+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:32.378075+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:25.443004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:23.634325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.175620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.608100+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.434930+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:20.501276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:13.348208+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.326775+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.748721+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.563588+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:27.752979+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:34:20.712532+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:32.885128+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[10,8]}],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":25,"length":1},{"begin":31,"length":1},{"begin":41,"length":1},{"begin":44,"length":1},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":58,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:34:25.934 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T01:34:25.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:34:26.213 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:34:26.214 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:34:26.224 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-15T01:34:25.095499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":5,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":12773700,"kb_used_data":1486100956,"kb_used_omap":52112,"kb_used_meta":0,"kb_avail":1486100156,"statfs":{"total":1534846828544,"available":1521766559744,"internally_reserved":819200,"allocated":1521767378944,"data_stored":1521767378944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53363571,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001207"},"pg_stats":[{"pgid":"1.0","version":"106'114","reported_seq":266,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888285+0000","last_change":"2024-09-15T01:33:49.902092+0000","last_active":"2024-09-15T01:34:19.888285+0000","last_peered":"2024-09-15T01:34:19.888285+0000","last_clean":"2024-09-15T01:34:19.888285+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:34:19.888285+0000","last_undegraded":"2024-09-15T01:34:19.888285+0000","last_fullsized":"2024-09-15T01:34:19.888285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'114","last_scrub_stamp":"2024-09-15T01:33:49.902036+0000","last_deep_scrub":"106'114","last_deep_scrub_stamp":"2024-09-15T01:33:49.902036+0000","last_clean_scrub_stamp":"2024-09-15T01:33:49.902036+0000","objects_scrubbed":2,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:12.193331+0000","scrub_duration":0.010874557,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"105'2","reported_seq":128,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.341551+0000","last_change":"2024-09-15T01:33:48.491848+0000","last_active":"2024-09-15T01:34:19.341551+0000","last_peered":"2024-09-15T01:34:19.341551+0000","last_clean":"2024-09-15T01:34:19.341551+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:34:19.341551+0000","last_undegraded":"2024-09-15T01:34:19.341551+0000","last_fullsized":"2024-09-15T01:34:19.341551+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'2","last_scrub_stamp":"2024-09-15T01:33:48.491771+0000","last_deep_scrub":"105'2","last_deep_scrub_stamp":"2024-09-15T01:33:48.491771+0000","last_clean_scrub_stamp":"2024-09-15T01:33:48.491771+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:01.566416+0000","scrub_duration":0.0025105769999999999,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.794589+0000","last_change":"2024-09-15T01:33:46.379860+0000","last_active":"2024-09-15T01:34:20.794589+0000","last_peered":"2024-09-15T01:34:20.794589+0000","last_clean":"2024-09-15T01:34:20.794589+0000","last_became_active":"2024-09-15T01:33:23.083242+0000","last_became_peered":"2024-09-15T01:33:23.083242+0000","last_unstale":"2024-09-15T01:34:20.794589+0000","last_undegraded":"2024-09-15T01:34:20.794589+0000","last_fullsized":"2024-09-15T01:34:20.794589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:46.379809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:46.379809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:49.087667+0000","scrub_duration":0.001946382,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8],"acting":[10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888248+0000","last_change":"2024-09-15T01:33:50.941845+0000","last_active":"2024-09-15T01:34:19.888248+0000","last_peered":"2024-09-15T01:34:19.888248+0000","last_clean":"2024-09-15T01:34:19.888248+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:34:19.888248+0000","last_undegraded":"2024-09-15T01:34:19.888248+0000","last_fullsized":"2024-09-15T01:34:19.888248+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:50.941790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:50.941790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:10.537027+0000","scrub_duration":0.0017165430000000001,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":133,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.484672+0000","last_change":"2024-09-15T01:33:46.870657+0000","last_active":"2024-09-15T01:34:19.484672+0000","last_peered":"2024-09-15T01:34:19.484672+0000","last_clean":"2024-09-15T01:34:19.484672+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:34:19.484672+0000","last_undegraded":"2024-09-15T01:34:19.484672+0000","last_fullsized":"2024-09-15T01:34:19.484672+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-15T01:33:46.870584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:46.870584+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:14.306792+0000","scrub_duration":0.0027185820000000002,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888035+0000","last_change":"2024-09-15T01:33:41.548849+0000","last_active":"2024-09-15T01:34:19.888035+0000","last_peered":"2024-09-15T01:34:19.888035+0000","last_clean":"2024-09-15T01:34:19.888035+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:34:19.888035+0000","last_undegraded":"2024-09-15T01:34:19.888035+0000","last_fullsized":"2024-09-15T01:34:19.888035+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:41.548799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:41.548799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:55.947522+0000","scrub_duration":0.001258681,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888166+0000","last_change":"2024-09-15T01:33:41.543896+0000","last_active":"2024-09-15T01:34:19.888166+0000","last_peered":"2024-09-15T01:34:19.888166+0000","last_clean":"2024-09-15T01:34:19.888166+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:34:19.888166+0000","last_undegraded":"2024-09-15T01:34:19.888166+0000","last_fullsized":"2024-09-15T01:34:19.888166+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:41.543850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:41.543850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:56.945346+0000","scrub_duration":0.0015955419999999999,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.587067+0000","last_change":"2024-09-15T01:33:39.638452+0000","last_active":"2024-09-15T01:34:20.587067+0000","last_peered":"2024-09-15T01:34:20.587067+0000","last_clean":"2024-09-15T01:34:20.587067+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:34:20.587067+0000","last_undegraded":"2024-09-15T01:34:20.587067+0000","last_fullsized":"2024-09-15T01:34:20.587067+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:39.638404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:39.638404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:59.511423+0000","scrub_duration":0.001362835,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":129,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.794259+0000","last_change":"2024-09-15T01:33:38.175676+0000","last_active":"2024-09-15T01:34:20.794259+0000","last_peered":"2024-09-15T01:34:20.794259+0000","last_clean":"2024-09-15T01:34:20.794259+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:34:20.794259+0000","last_undegraded":"2024-09-15T01:34:20.794259+0000","last_fullsized":"2024-09-15T01:34:20.794259+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:38.175625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:38.175625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:44.144155+0000","scrub_duration":0.001339526,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881456,"kb_used_omap":3954,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":65536,"allocated":95110610944,"data_stored":95110610944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4049435,"internal_metadata":0},"hb_peers":[1,2,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":16.196999999999999,"5min":27.753,"15min":29.678999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":299.721,"5min":574.26900000000001,"15min":574.26900000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":18.404,"5min":4.0270000000000001,"15min":1.631},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":342.08300000000003,"5min":342.08300000000003,"15min":342.08300000000003},"last":0.35799999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:33:20 2024","interfaces":[{"interface":"back","average":{"1min":32.810000000000002,"5min":32.810000000000002,"15min":32.810000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":644.97699999999998,"5min":644.97699999999998,"15min":644.97699999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":805.39499999999998}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.71399999999999997,"15min":0.73999999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.69299999999999995,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":11.887,"5min":2.8730000000000002,"15min":1.371},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":214.994,"5min":214.994,"15min":214.994},"last":0.66900000000000004}]},{"osd":5,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47699999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.502}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82899999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.621},{"interface":"front","average":{"1min":10.272,"5min":2.625,"15min":1.351},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":173.11099999999999,"5min":173.11099999999999,"15min":173.11099999999999},"last":0.48599999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":6.056,"15min":6.9420000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.589,"5min":134.12200000000001,"15min":134.12200000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":45.521000000000001,"5min":9.7249999999999996,"15min":3.7589999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":846.97400000000005,"5min":846.97400000000005,"15min":846.97400000000005},"last":0.65400000000000003}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.56599999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.5649999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":17.137,"5min":4.0940000000000003,"15min":1.9199999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":313.495,"5min":313.495,"15min":313.495},"last":0.72499999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.69399999999999995}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81399999999999995,"15min":0.81299999999999994},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.70899999999999996}]}]},{"osd":9,"up_from":21,"seq":90194313244,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881452,"kb_used_omap":1548,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":61440,"allocated":95110606848,"data_stored":95110606848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1586173,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.46999999999999997},"min":{"1min":0.38700000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.49299999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":23.486999999999998,"5min":5.2149999999999999,"15min":2.1699999999999999},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":462.93700000000001,"5min":462.93700000000001,"15min":462.93700000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.94399999999999995,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84399999999999997,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89800000000000002,"5min":1.97,"15min":1.97},"last":0.93100000000000005}]},{"osd":8,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.0720000000000001,"15min":1.1519999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":10.916,"15min":10.916},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.623,"5min":1.218,"15min":1.3169999999999999},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":13.147,"15min":13.147},"last":0.59499999999999997}]},{"osd":10,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":5.8819999999999997,"5min":1.702,"15min":1.006},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":127.033,"5min":127.033,"15min":127.033},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.90600000000000003,"15min":0.95299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92900000000000005,"5min":6.4539999999999997,"15min":6.4539999999999997},"last":0.877}]},{"osd":11,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":10.221,"5min":2.5840000000000001,"15min":1.3109999999999999},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":229.28899999999999,"5min":229.28899999999999,"15min":229.28899999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.88700000000000001,"15min":0.93100000000000005},"min":{"1min":0.42199999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.874,"5min":6.9489999999999998,"15min":6.9489999999999998},"last":0.68600000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":23.875,"15min":27.75},"min":{"1min":0.47899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97699999999999998,"5min":495.95800000000003,"15min":495.95800000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":43.860999999999997,"15min":51.058999999999997},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":914.18499999999995,"15min":914.18499999999995},"last":0.94999999999999996}]},{"osd":13,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":2.7599999999999998,"5min":1.595,"15min":1.401},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":49.841000000000001,"5min":49.841000000000001,"15min":49.841000000000001},"last":0.84699999999999998}]},{"osd":14,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86099999999999999,"5min":1.96,"15min":1.96},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.96199999999999997}]},{"osd":15,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98799999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91000000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.66300000000000003}]}]},{"osd":3,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798228,"kb_used_data":92881448,"kb_used_omap":2127,"kb_used_meta":0,"kb_avail":92881388,"statfs":{"total":95927926784,"available":95110541312,"internally_reserved":61440,"allocated":95110602752,"data_stored":95110602752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2179043,"internal_metadata":0},"hb_peers":[0,2,4,6,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.365,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.91700000000000004}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.57699999999999996,"5min":0.312,"15min":0.312},"max":{"1min":0.99099999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.503}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.81399999999999995,"15min":0.84699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1850000000000001,"5min":7.4720000000000004,"15min":7.4720000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":7.5629999999999997,"15min":8.7230000000000008},"min":{"1min":0.41099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.1100000000000001,"5min":182.87,"15min":182.87},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.5,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":3.202},{"interface":"front","average":{"1min":28.616,"5min":6.2119999999999997,"15min":2.4780000000000002},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":501.65800000000002,"5min":501.65800000000002,"15min":501.65800000000002},"last":0.56200000000000006}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":24.288,"5min":24.288,"15min":24.288},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":500.678,"5min":500.678,"15min":500.678},"last":0.55100000000000005},{"interface":"front","average":{"1min":35.298999999999999,"5min":35.298999999999999,"15min":35.298999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":500.79899999999998,"5min":500.79899999999998,"15min":500.79899999999998},"last":0.52200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":11.316000000000001,"5min":11.316000000000001,"15min":11.316000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":200.40799999999999,"5min":200.40799999999999,"15min":200.40799999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.59799999999999998}]},{"osd":11,"last update":"Sun Sep 15 01:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.433}]},{"osd":12,"last update":"Sun Sep 15 01:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.61199999999999999}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.78400000000000003,"15min":0.76800000000000002},"min":{"1min":0.67900000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.427,"5min":1.637,"15min":1.637},"last":0.624},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0169999999999999,"15min":1.0389999999999999},"min":{"1min":0.63200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.476,"5min":7.4500000000000002,"15min":7.4500000000000002},"last":0.57999999999999996}]},{"osd":14,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53200000000000003}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.52200000000000002}]}]},{"osd":0,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798352,"kb_used_data":92881324,"kb_used_omap":5384,"kb_used_meta":0,"kb_avail":92881264,"statfs":{"total":95927926784,"available":95110414336,"internally_reserved":61440,"allocated":95110475776,"data_stored":95110475776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5513250,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.872},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.113,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.113}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":31.643999999999998,"15min":36.808999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.488,"5min":814.68299999999999,"15min":814.68299999999999},"last":0.624},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.63700000000000001,"15min":0.65200000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1890000000000001,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":40.415999999999997,"5min":8.5380000000000003,"15min":3.2250000000000001},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":799.04999999999995,"5min":799.04999999999995,"15min":799.04999999999995},"last":0.66500000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69399999999999995,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.2769999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.264,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.133}]},{"osd":7,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":46.667999999999999,"5min":9.8330000000000002,"15min":3.694},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":920.58399999999995,"5min":920.58399999999995,"15min":920.58399999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.41599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0629999999999999,"5min":1.304,"15min":1.304},"last":0.84799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.222,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.222}]},{"osd":10,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.78500000000000003}]},{"osd":12,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":1.026},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.58899999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":25.651,"5min":25.651,"15min":25.651},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":501.07400000000001,"5min":501.07400000000001,"15min":501.07400000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":44.5,"5min":44.5,"15min":44.5},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":501.08800000000002,"5min":501.08800000000002,"15min":501.08800000000002},"last":1.2070000000000001}]},{"osd":15,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.403,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.96999999999999997}]}]},{"osd":13,"up_from":21,"seq":90194313245,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799440,"kb_used_data":92880240,"kb_used_omap":4484,"kb_used_meta":0,"kb_avail":92880176,"statfs":{"total":95927926784,"available":95109300224,"internally_reserved":65536,"allocated":95109365760,"data_stored":95109365760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4592085,"internal_metadata":0},"hb_peers":[2,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.1129999999999995,"5min":8.1129999999999995,"15min":8.1129999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":8.1430000000000007,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":0.60899999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":19.710000000000001,"5min":19.710000000000001,"15min":19.710000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":342.56099999999998,"5min":342.56099999999998,"15min":342.56099999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":28.227,"5min":28.227,"15min":28.227},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":495.40800000000002,"5min":495.40800000000002,"15min":495.40800000000002},"last":0.60099999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.7960000000000003,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.7619999999999996,"5min":5.7619999999999996,"15min":5.7619999999999996},"last":0.64600000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.373}]},{"osd":6,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":18.945,"5min":18.945,"15min":18.945},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":293.34800000000001,"5min":293.34800000000001,"15min":293.34800000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.40100000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.4779999999999998,"5min":6.4779999999999998,"15min":6.4779999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":5.726,"5min":5.726,"15min":5.726},"last":0.73199999999999998}]},{"osd":8,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":44.985999999999997,"5min":44.985999999999997,"15min":44.985999999999997},"last":0.52200000000000002}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.63100000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54000000000000004}]},{"osd":14,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.70899999999999996}]},{"osd":15,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.746}]}]},{"osd":2,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798928,"kb_used_data":92880748,"kb_used_omap":4392,"kb_used_meta":0,"kb_avail":92880688,"statfs":{"total":95927926784,"available":95109824512,"internally_reserved":61440,"allocated":95109885952,"data_stored":95109885952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4498429,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95199999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86599999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.38400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":46.487000000000002,"5min":9.6280000000000001,"15min":3.4849999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":829.03399999999999,"5min":829.03399999999999,"15min":829.03399999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.43099999999999999,"15min":0.41099999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.72899999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":39.168999999999997,"5min":13.167999999999999,"15min":8.8339999999999996},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":655.90899999999999,"5min":655.90899999999999,"15min":655.90899999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61299999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0329999999999999,"5min":1.329,"15min":1.329},"last":0.79700000000000004},{"interface":"front","average":{"1min":54.841000000000001,"5min":11.481,"15min":4.2539999999999996},"min":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":976.00800000000004,"5min":976.00800000000004,"15min":976.00800000000004},"last":0.74399999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63}]},{"osd":11,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.68700000000000006,"15min":0.63600000000000001},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":6.4240000000000004,"5min":6.4240000000000004,"15min":6.4240000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.74399999999999999,"15min":0.68999999999999995},"min":{"1min":0.47299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":6.4390000000000001,"5min":6.4390000000000001,"15min":6.4390000000000001},"last":0.89800000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":47.031999999999996,"5min":9.9469999999999992,"15min":3.766},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":834.505,"5min":834.505,"15min":834.505},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97199999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.71599999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99199999999999999,"5min":1.363,"15min":1.363},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.88}]},{"osd":14,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.92200000000000004}]}]},{"osd":15,"up_from":21,"seq":90194313246,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798156,"kb_used_data":92881472,"kb_used_omap":3194,"kb_used_meta":0,"kb_avail":92881460,"statfs":{"total":95927926784,"available":95110615040,"internally_reserved":12288,"allocated":95110627328,"data_stored":95110627328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3271314,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.79300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":12.098000000000001,"5min":2.8300000000000001,"15min":1.2849999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":162.16800000000001,"5min":162.16800000000001,"15min":162.16800000000001},"last":0.76500000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":47.701000000000001,"5min":9.8719999999999999,"15min":3.5670000000000002},"min":{"1min":0.26600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":662.84000000000003,"5min":662.84000000000003,"15min":662.84000000000003},"last":0.39700000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64200000000000002,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57099999999999995,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.40400000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94899999999999995,"5min":1.385,"15min":1.385},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.85699999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":4.798,"15min":5.5069999999999997},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69999999999999996,"5min":90.775999999999996,"15min":90.775999999999996},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Sep 15 01:33:36 2024","interfaces":[{"interface":"back","average":{"1min":50.158999999999999,"5min":50.158999999999999,"15min":50.158999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":894.27700000000004,"5min":894.27700000000004,"15min":894.27700000000004},"last":0.497},{"interface":"front","average":{"1min":49.308,"5min":49.308,"15min":49.308},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":878.12199999999996,"5min":878.12199999999996,"15min":878.12199999999996},"last":0.61299999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":50.171999999999997,"5min":10.541,"15min":3.9359999999999999},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":645.34900000000005,"5min":645.34900000000005,"15min":645.34900000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":27.992000000000001,"15min":32.540999999999997},"min":{"1min":0.45900000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99099999999999999,"5min":542.06600000000003,"15min":542.06600000000003},"last":0.59699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":1.603,"15min":1.603},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96199999999999997,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.78200000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":1.163,"15min":1.163},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75,"5min":12.249000000000001,"15min":14.164999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.224,"5min":246.56100000000001,"15min":246.56100000000001},"last":0.81599999999999995}]}]},{"osd":4,"up_from":21,"seq":90194313245,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798196,"kb_used_data":92881444,"kb_used_omap":3058,"kb_used_meta":0,"kb_avail":92881420,"statfs":{"total":95927926784,"available":95110574080,"internally_reserved":24576,"allocated":95110598656,"data_stored":95110598656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3131934,"internal_metadata":0},"hb_peers":[0,3,5,6,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":17.896999999999998,"5min":17.896999999999998,"15min":17.896999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":314.661,"5min":314.661,"15min":314.661},"last":0.42499999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51300000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.747},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":1.127}]},{"osd":3,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.55900000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0940000000000001,"5min":1.179,"15min":1.179},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.45000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.622},{"interface":"front","average":{"1min":42.378,"5min":24.170000000000002,"15min":21.135000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":918.75699999999995,"5min":918.75699999999995,"15min":918.75699999999995},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":7.5110000000000001,"5min":2.1309999999999998,"15min":1.2350000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":142.26599999999999,"5min":142.26599999999999,"15min":142.26599999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.57899999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1719999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0840000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.871}]},{"osd":9,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78300000000000003,"15min":0.77900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.59499999999999997}]},{"osd":10,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":40.198999999999998,"5min":8.7089999999999996,"15min":3.4609999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":828.54499999999996,"5min":828.54499999999996,"15min":828.54499999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":14.634,"5min":3.5920000000000001,"15min":1.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":291.08800000000002,"5min":291.08800000000002,"15min":291.08800000000002},"last":0.67800000000000005}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":28.721,"5min":28.721,"15min":28.721},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":558.56700000000001,"5min":558.56700000000001,"15min":558.56700000000001},"last":0.57999999999999996}]},{"osd":12,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1799999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.5}]},{"osd":13,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.71699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.61099999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3169999999999999,"5min":1.325,"15min":1.325},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.90500000000000003,"15min":0.90700000000000003},"min":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.256,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.64300000000000002}]}]},{"osd":5,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798296,"kb_used_data":92881368,"kb_used_omap":3055,"kb_used_meta":0,"kb_avail":92881320,"statfs":{"total":95927926784,"available":95110471680,"internally_reserved":49152,"allocated":95110520832,"data_stored":95110520832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3128858,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.499},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.70799999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.88300000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72599999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.628,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.48,"5min":1.524,"15min":1.524},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.70199999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.80800000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92200000000000004,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.60599999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":36.902999999999999,"5min":7.8339999999999996,"15min":2.9889999999999999},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":764.33399999999995,"5min":764.33399999999995,"15min":764.33399999999995},"last":0.441},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.39200000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1899999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":17.335999999999999,"5min":22.786000000000001,"15min":23.693999999999999},"min":{"1min":0.496,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":346.49200000000002,"5min":492.697,"15min":492.697},"last":0.78800000000000003}]},{"osd":15,"last update":"Sun Sep 15 01:33:26 2024","interfaces":[{"interface":"back","average":{"1min":13.939,"5min":13.939,"15min":13.939},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":291.12599999999998,"5min":291.12599999999998,"15min":291.12599999999998},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.93700000000000006}]}]},{"osd":6,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798300,"kb_used_data":92881364,"kb_used_omap":3133,"kb_used_meta":0,"kb_avail":92881316,"statfs":{"total":95927926784,"available":95110467584,"internally_reserved":49152,"allocated":95110516736,"data_stored":95110516736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3208697,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.85699999999999998,"15min":0.88100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.165,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77600000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.878,"5min":1.264,"15min":1.264},"last":0.60499999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.53800000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77800000000000002,"5min":1.163,"15min":1.163},"last":0.755}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.379,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83799999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.746,"15min":0.746},"last":0.49399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.90200000000000002}]},{"osd":10,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":61.164000000000001,"5min":12.723000000000001,"15min":4.649},"min":{"1min":0.48099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":656.21000000000004,"5min":656.21000000000004,"15min":656.21000000000004},"last":0.83399999999999996}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":11.742000000000001,"5min":2.8580000000000001,"15min":1.3779999999999999},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":200.52799999999999,"5min":200.52799999999999,"15min":200.52799999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.64600000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82999999999999996,"5min":1.353,"15min":1.353},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.99199999999999999,"15min":1.036},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97999999999999998,"5min":5.8470000000000004,"15min":5.8470000000000004},"last":0.85799999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.55800000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.55200000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.034,"5min":1.212,"15min":1.212},"last":0.91500000000000004}]}]},{"osd":7,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881452,"kb_used_omap":2577,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":61440,"allocated":95110606848,"data_stored":95110606848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2639163,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.48199999999999998,"15min":0.47099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":47.609999999999999,"15min":55.453000000000003},"min":{"1min":0.36699999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.69999999999999996,"5min":874.39499999999998,"15min":874.39499999999998},"last":0.38300000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":4.7350000000000003,"5min":4.7350000000000003,"15min":4.7350000000000003},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.33700000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.58299999999999996}]},{"osd":6,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":1.165,"15min":1.165},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":1.0880000000000001,"15min":1.167},"min":{"1min":0.42799999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":5.6539999999999999,"15min":5.6539999999999999},"last":0.52400000000000002}]},{"osd":8,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55900000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":4.4359999999999999,"15min":5.069},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84899999999999998,"5min":62.991999999999997,"15min":62.991999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.50700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67600000000000005,"15min":0.68400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":1.21,"15min":1.21},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":43.418999999999997,"15min":50.549999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93400000000000005,"5min":792.73599999999999,"15min":792.73599999999999},"last":0.47299999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.091,"15min":1.157},"min":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92100000000000004,"5min":8.1489999999999991,"15min":8.1489999999999991},"last":0.68999999999999995}]},{"osd":14,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.47199999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.878,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":41.905000000000001,"15min":48.776000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.83099999999999996,"5min":773.18799999999999,"15min":773.18799999999999},"last":0.72699999999999998}]},{"osd":15,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.1220000000000001,"15min":1.216},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79000000000000004,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83399999999999996,"5min":1.018,"15min":1.018},"last":0.60599999999999998}]}]},{"osd":8,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798352,"kb_used_data":92881324,"kb_used_omap":3898,"kb_used_meta":0,"kb_avail":92881264,"statfs":{"total":95927926784,"available":95110414336,"internally_reserved":61440,"allocated":95110475776,"data_stored":95110475776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3991961,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.373},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.626,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.40500000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.503},{"interface":"front","average":{"1min":24.794,"5min":24.794,"15min":24.794},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":413.66800000000001,"5min":413.66800000000001,"15min":413.66800000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55300000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.44800000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72499999999999998,"5min":1.093,"15min":1.093},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.44700000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.95599999999999996,"5min":1.073,"15min":1.073},"last":0.57499999999999996}]},{"osd":7,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":30.347000000000001,"5min":6.5869999999999997,"15min":2.6269999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":505.29300000000001,"5min":505.29300000000001,"15min":505.29300000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":12.156000000000001,"5min":2.9510000000000001,"15min":1.417},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":196.00299999999999,"5min":196.00299999999999,"15min":196.00299999999999},"last":0.79900000000000004}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.46899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97699999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.75600000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0249999999999999,"5min":1.208,"15min":1.208},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.004,"5min":1.351,"15min":1.351},"last":0.84799999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.503,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66100000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":1.054,"15min":1.054},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98999999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.67500000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798172,"kb_used_data":92881492,"kb_used_omap":3280,"kb_used_meta":0,"kb_avail":92881444,"statfs":{"total":95927926784,"available":95110598656,"internally_reserved":49152,"allocated":95110647808,"data_stored":95110647808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3358734,"internal_metadata":0},"hb_peers":[3,5,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.899},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75700000000000001,"15min":0.752},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":3.9750000000000001,"5min":3.9750000000000001,"15min":3.9750000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":3.9039999999999999,"5min":4.3150000000000004,"15min":4.383},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":56.188000000000002,"5min":64.938999999999993,"15min":64.938999999999993},"last":0.84799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":4.7960000000000003,"5min":1.5009999999999999,"15min":0.95199999999999996},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":75.778999999999996,"5min":75.778999999999996,"15min":75.778999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.75900000000000001,"15min":0.78100000000000003},"min":{"1min":0.45300000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84599999999999997,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.47999999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":44.508000000000003,"15min":51.82},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96299999999999997,"5min":500.92099999999999,"15min":500.92099999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.41399999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.83699999999999997}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.52500000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84199999999999997,"15min":0.87},"min":{"1min":0.503,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0229999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.82299999999999995}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.4460000000000002},{"interface":"front","average":{"1min":36.435000000000002,"5min":36.435000000000002,"15min":36.435000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":641.649,"5min":641.649,"15min":641.649},"last":2.4740000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.048},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.92800000000000005}]},{"osd":15,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.93899999999999995,"15min":0.97299999999999998},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.036,"5min":6.0670000000000002,"15min":6.0670000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0089999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.94799999999999995}]}]},{"osd":11,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798032,"kb_used_data":92881596,"kb_used_omap":3335,"kb_used_meta":0,"kb_avail":92881584,"statfs":{"total":95927926784,"available":95110742016,"internally_reserved":12288,"allocated":95110754304,"data_stored":95110754304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3415222,"internal_metadata":0},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":46.469000000000001,"5min":46.469000000000001,"15min":46.469000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":873.95500000000004,"5min":873.95500000000004,"15min":873.95500000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.954,"5min":1.954,"15min":1.954},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":28.431999999999999,"5min":28.431999999999999,"15min":28.431999999999999},"last":0.50900000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":565.90700000000004}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":13.093,"5min":3.125,"15min":1.4630000000000001},"min":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":250.767,"5min":250.767,"15min":250.767},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84899999999999998,"5min":0.871,"15min":0.871},"last":0.495}]},{"osd":5,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.623}]},{"osd":6,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.30199999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.433},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47699999999999998}]},{"osd":9,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51300000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86899999999999999,"5min":1.042,"15min":1.042},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90200000000000002,"5min":2.262,"15min":2.262},"last":0.39500000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.79000000000000004,"15min":0.81799999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93400000000000005,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.82099999999999995,"15min":0.84699999999999998},"min":{"1min":0.45300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85999999999999999,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":0.58599999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":1.903,"5min":0.95599999999999996,"15min":0.79800000000000004},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":25.056999999999999,"5min":25.056999999999999,"15min":25.056999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.442,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.54200000000000004}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":4.4009999999999998,"5min":13.781000000000001,"15min":15.343999999999999},"min":{"1min":0.51200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":74.078000000000003,"5min":310.78199999999998,"15min":310.78199999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.436,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97099999999999997,"5min":2.238,"15min":2.238},"last":0.52400000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":12,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798284,"kb_used_data":92881380,"kb_used_omap":1469,"kb_used_meta":0,"kb_avail":92881332,"statfs":{"total":95927926784,"available":95110483968,"internally_reserved":49152,"allocated":95110533120,"data_stored":95110533120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1505179,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.28000000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":8.0050000000000008,"5min":8.0050000000000008,"15min":8.0050000000000008},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":0.35599999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.47099999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.51500000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":24.937000000000001,"5min":24.937000000000001,"15min":24.937000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":488.69,"5min":488.69,"15min":488.69},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5419999999999998,"5min":3.5419999999999998,"15min":3.5419999999999998},"last":0.29599999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.6309999999999998,"5min":3.6309999999999998,"15min":3.6309999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.57899999999999996}]},{"osd":11,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.499}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.433},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.40400000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.54300000000000004}]},{"osd":15,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.56899999999999995}]}]},{"osd":1,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798292,"kb_used_data":92881396,"kb_used_omap":3216,"kb_used_meta":0,"kb_avail":92881324,"statfs":{"total":95927926784,"available":95110475776,"internally_reserved":73728,"allocated":95110549504,"data_stored":95110549504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3294094,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":55.167999999999999,"5min":11.429,"15min":4.1390000000000002},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.31},"max":{"1min":875.06100000000004,"5min":875.06100000000004,"15min":875.06100000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.754,"15min":0.754},"last":0.504}]},{"osd":2,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79100000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61299999999999999,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91100000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59099999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.79300000000000004}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.29999999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79200000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":28.789000000000001,"15min":33.491},"min":{"1min":0.38400000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82999999999999996,"5min":635.46400000000006,"15min":635.46400000000006},"last":0.41999999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.53800000000000003},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71199999999999997,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.80500000000000005,"15min":0.85699999999999998},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72699999999999998,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.69799999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":54.337000000000003,"5min":11.387,"15min":4.2290000000000001},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":806.31200000000001,"5min":806.31200000000001,"15min":806.31200000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68000000000000005,"15min":0.68600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83799999999999997,"5min":1.341,"15min":1.341},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.75,"15min":0.751},"min":{"1min":0.52600000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.4379999999999999,"5min":1.825,"15min":1.825},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.48499999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0469999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.82499999999999996}]},{"osd":8,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":4.0830000000000002,"5min":1.395,"15min":0.94699999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":55.694000000000003,"5min":55.694000000000003,"15min":55.694000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69199999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.88900000000000001,"5min":1.492,"15min":1.492},"last":0.65500000000000003}]},{"osd":10,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":2.149,"15min":2.222},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":17.527000000000001,"5min":27.920000000000002,"15min":27.920000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":8.1069999999999993,"15min":9.3320000000000007},"min":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.026,"5min":166.16200000000001,"15min":166.16200000000001},"last":0.80800000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.755},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.496,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84999999999999998,"5min":1.393,"15min":1.393},"last":0.60999999999999999}]}]}],"pool_statfs":[]}} 2024-09-15T01:34:26.228 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:34:26.523 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-15T01:34:25.095499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":5,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":12773700,"kb_used_data":1486100956,"kb_used_omap":52112,"kb_used_meta":0,"kb_avail":1486100156,"statfs":{"total":1534846828544,"available":1521766559744,"internally_reserved":819200,"allocated":1521767378944,"data_stored":1521767378944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53363571,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001207"},"pg_stats":[{"pgid":"1.0","version":"106'114","reported_seq":266,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888285+0000","last_change":"2024-09-15T01:33:49.902092+0000","last_active":"2024-09-15T01:34:19.888285+0000","last_peered":"2024-09-15T01:34:19.888285+0000","last_clean":"2024-09-15T01:34:19.888285+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:34:19.888285+0000","last_undegraded":"2024-09-15T01:34:19.888285+0000","last_fullsized":"2024-09-15T01:34:19.888285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'114","last_scrub_stamp":"2024-09-15T01:33:49.902036+0000","last_deep_scrub":"106'114","last_deep_scrub_stamp":"2024-09-15T01:33:49.902036+0000","last_clean_scrub_stamp":"2024-09-15T01:33:49.902036+0000","objects_scrubbed":2,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:12.193331+0000","scrub_duration":0.010874557,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"105'2","reported_seq":128,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.341551+0000","last_change":"2024-09-15T01:33:48.491848+0000","last_active":"2024-09-15T01:34:19.341551+0000","last_peered":"2024-09-15T01:34:19.341551+0000","last_clean":"2024-09-15T01:34:19.341551+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:34:19.341551+0000","last_undegraded":"2024-09-15T01:34:19.341551+0000","last_fullsized":"2024-09-15T01:34:19.341551+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'2","last_scrub_stamp":"2024-09-15T01:33:48.491771+0000","last_deep_scrub":"105'2","last_deep_scrub_stamp":"2024-09-15T01:33:48.491771+0000","last_clean_scrub_stamp":"2024-09-15T01:33:48.491771+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:01.566416+0000","scrub_duration":0.0025105769999999999,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.794589+0000","last_change":"2024-09-15T01:33:46.379860+0000","last_active":"2024-09-15T01:34:20.794589+0000","last_peered":"2024-09-15T01:34:20.794589+0000","last_clean":"2024-09-15T01:34:20.794589+0000","last_became_active":"2024-09-15T01:33:23.083242+0000","last_became_peered":"2024-09-15T01:33:23.083242+0000","last_unstale":"2024-09-15T01:34:20.794589+0000","last_undegraded":"2024-09-15T01:34:20.794589+0000","last_fullsized":"2024-09-15T01:34:20.794589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:46.379809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:46.379809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:49.087667+0000","scrub_duration":0.001946382,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8],"acting":[10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888248+0000","last_change":"2024-09-15T01:33:50.941845+0000","last_active":"2024-09-15T01:34:19.888248+0000","last_peered":"2024-09-15T01:34:19.888248+0000","last_clean":"2024-09-15T01:34:19.888248+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:34:19.888248+0000","last_undegraded":"2024-09-15T01:34:19.888248+0000","last_fullsized":"2024-09-15T01:34:19.888248+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:50.941790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:50.941790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:10.537027+0000","scrub_duration":0.0017165430000000001,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":133,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.484672+0000","last_change":"2024-09-15T01:33:46.870657+0000","last_active":"2024-09-15T01:34:19.484672+0000","last_peered":"2024-09-15T01:34:19.484672+0000","last_clean":"2024-09-15T01:34:19.484672+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:34:19.484672+0000","last_undegraded":"2024-09-15T01:34:19.484672+0000","last_fullsized":"2024-09-15T01:34:19.484672+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-15T01:33:46.870584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:46.870584+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:14.306792+0000","scrub_duration":0.0027185820000000002,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888035+0000","last_change":"2024-09-15T01:33:41.548849+0000","last_active":"2024-09-15T01:34:19.888035+0000","last_peered":"2024-09-15T01:34:19.888035+0000","last_clean":"2024-09-15T01:34:19.888035+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:34:19.888035+0000","last_undegraded":"2024-09-15T01:34:19.888035+0000","last_fullsized":"2024-09-15T01:34:19.888035+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:41.548799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:41.548799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:55.947522+0000","scrub_duration":0.001258681,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888166+0000","last_change":"2024-09-15T01:33:41.543896+0000","last_active":"2024-09-15T01:34:19.888166+0000","last_peered":"2024-09-15T01:34:19.888166+0000","last_clean":"2024-09-15T01:34:19.888166+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:34:19.888166+0000","last_undegraded":"2024-09-15T01:34:19.888166+0000","last_fullsized":"2024-09-15T01:34:19.888166+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:41.543850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:41.543850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:56.945346+0000","scrub_duration":0.0015955419999999999,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.587067+0000","last_change":"2024-09-15T01:33:39.638452+0000","last_active":"2024-09-15T01:34:20.587067+0000","last_peered":"2024-09-15T01:34:20.587067+0000","last_clean":"2024-09-15T01:34:20.587067+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:34:20.587067+0000","last_undegraded":"2024-09-15T01:34:20.587067+0000","last_fullsized":"2024-09-15T01:34:20.587067+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:39.638404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:39.638404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:59.511423+0000","scrub_duration":0.001362835,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":129,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.794259+0000","last_change":"2024-09-15T01:33:38.175676+0000","last_active":"2024-09-15T01:34:20.794259+0000","last_peered":"2024-09-15T01:34:20.794259+0000","last_clean":"2024-09-15T01:34:20.794259+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:34:20.794259+0000","last_undegraded":"2024-09-15T01:34:20.794259+0000","last_fullsized":"2024-09-15T01:34:20.794259+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:38.175625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:38.175625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:44.144155+0000","scrub_duration":0.001339526,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881456,"kb_used_omap":3954,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":65536,"allocated":95110610944,"data_stored":95110610944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4049435,"internal_metadata":0},"hb_peers":[1,2,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":16.196999999999999,"5min":27.753,"15min":29.678999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":299.721,"5min":574.26900000000001,"15min":574.26900000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":18.404,"5min":4.0270000000000001,"15min":1.631},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":342.08300000000003,"5min":342.08300000000003,"15min":342.08300000000003},"last":0.35799999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:33:20 2024","interfaces":[{"interface":"back","average":{"1min":32.810000000000002,"5min":32.810000000000002,"15min":32.810000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":644.97699999999998,"5min":644.97699999999998,"15min":644.97699999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":805.39499999999998}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.71399999999999997,"15min":0.73999999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.69299999999999995,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":11.887,"5min":2.8730000000000002,"15min":1.371},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":214.994,"5min":214.994,"15min":214.994},"last":0.66900000000000004}]},{"osd":5,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47699999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.502}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82899999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.621},{"interface":"front","average":{"1min":10.272,"5min":2.625,"15min":1.351},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":173.11099999999999,"5min":173.11099999999999,"15min":173.11099999999999},"last":0.48599999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":6.056,"15min":6.9420000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.589,"5min":134.12200000000001,"15min":134.12200000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":45.521000000000001,"5min":9.7249999999999996,"15min":3.7589999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":846.97400000000005,"5min":846.97400000000005,"15min":846.97400000000005},"last":0.65400000000000003}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.56599999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.5649999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":17.137,"5min":4.0940000000000003,"15min":1.9199999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":313.495,"5min":313.495,"15min":313.495},"last":0.72499999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.69399999999999995}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81399999999999995,"15min":0.81299999999999994},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.70899999999999996}]}]},{"osd":9,"up_from":21,"seq":90194313244,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881452,"kb_used_omap":1548,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":61440,"allocated":95110606848,"data_stored":95110606848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1586173,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.46999999999999997},"min":{"1min":0.38700000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.49299999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":23.486999999999998,"5min":5.2149999999999999,"15min":2.1699999999999999},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":462.93700000000001,"5min":462.93700000000001,"15min":462.93700000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.94399999999999995,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84399999999999997,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89800000000000002,"5min":1.97,"15min":1.97},"last":0.93100000000000005}]},{"osd":8,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.0720000000000001,"15min":1.1519999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":10.916,"15min":10.916},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.623,"5min":1.218,"15min":1.3169999999999999},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":13.147,"15min":13.147},"last":0.59499999999999997}]},{"osd":10,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":5.8819999999999997,"5min":1.702,"15min":1.006},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":127.033,"5min":127.033,"15min":127.033},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.90600000000000003,"15min":0.95299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92900000000000005,"5min":6.4539999999999997,"15min":6.4539999999999997},"last":0.877}]},{"osd":11,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":10.221,"5min":2.5840000000000001,"15min":1.3109999999999999},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":229.28899999999999,"5min":229.28899999999999,"15min":229.28899999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.88700000000000001,"15min":0.93100000000000005},"min":{"1min":0.42199999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.874,"5min":6.9489999999999998,"15min":6.9489999999999998},"last":0.68600000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":23.875,"15min":27.75},"min":{"1min":0.47899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97699999999999998,"5min":495.95800000000003,"15min":495.95800000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":43.860999999999997,"15min":51.058999999999997},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":914.18499999999995,"15min":914.18499999999995},"last":0.94999999999999996}]},{"osd":13,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":2.7599999999999998,"5min":1.595,"15min":1.401},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":49.841000000000001,"5min":49.841000000000001,"15min":49.841000000000001},"last":0.84699999999999998}]},{"osd":14,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86099999999999999,"5min":1.96,"15min":1.96},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.96199999999999997}]},{"osd":15,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98799999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91000000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.66300000000000003}]}]},{"osd":3,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798228,"kb_used_data":92881448,"kb_used_omap":2127,"kb_used_meta":0,"kb_avail":92881388,"statfs":{"total":95927926784,"available":95110541312,"internally_reserved":61440,"allocated":95110602752,"data_stored":95110602752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2179043,"internal_metadata":0},"hb_peers":[0,2,4,6,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.365,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.91700000000000004}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.57699999999999996,"5min":0.312,"15min":0.312},"max":{"1min":0.99099999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.503}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.81399999999999995,"15min":0.84699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1850000000000001,"5min":7.4720000000000004,"15min":7.4720000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":7.5629999999999997,"15min":8.7230000000000008},"min":{"1min":0.41099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.1100000000000001,"5min":182.87,"15min":182.87},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.5,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":3.202},{"interface":"front","average":{"1min":28.616,"5min":6.2119999999999997,"15min":2.4780000000000002},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":501.65800000000002,"5min":501.65800000000002,"15min":501.65800000000002},"last":0.56200000000000006}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":24.288,"5min":24.288,"15min":24.288},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":500.678,"5min":500.678,"15min":500.678},"last":0.55100000000000005},{"interface":"front","average":{"1min":35.298999999999999,"5min":35.298999999999999,"15min":35.298999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":500.79899999999998,"5min":500.79899999999998,"15min":500.79899999999998},"last":0.52200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":11.316000000000001,"5min":11.316000000000001,"15min":11.316000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":200.40799999999999,"5min":200.40799999999999,"15min":200.40799999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.59799999999999998}]},{"osd":11,"last update":"Sun Sep 15 01:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.433}]},{"osd":12,"last update":"Sun Sep 15 01:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.61199999999999999}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.78400000000000003,"15min":0.76800000000000002},"min":{"1min":0.67900000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.427,"5min":1.637,"15min":1.637},"last":0.624},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0169999999999999,"15min":1.0389999999999999},"min":{"1min":0.63200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.476,"5min":7.4500000000000002,"15min":7.4500000000000002},"last":0.57999999999999996}]},{"osd":14,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53200000000000003}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.52200000000000002}]}]},{"osd":0,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798352,"kb_used_data":92881324,"kb_used_omap":5384,"kb_used_meta":0,"kb_avail":92881264,"statfs":{"total":95927926784,"available":95110414336,"internally_reserved":61440,"allocated":95110475776,"data_stored":95110475776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5513250,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.872},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.113,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.113}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":31.643999999999998,"15min":36.808999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.488,"5min":814.68299999999999,"15min":814.68299999999999},"last":0.624},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.63700000000000001,"15min":0.65200000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1890000000000001,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":40.415999999999997,"5min":8.5380000000000003,"15min":3.2250000000000001},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":799.04999999999995,"5min":799.04999999999995,"15min":799.04999999999995},"last":0.66500000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69399999999999995,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.2769999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.264,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.133}]},{"osd":7,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":46.667999999999999,"5min":9.8330000000000002,"15min":3.694},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":920.58399999999995,"5min":920.58399999999995,"15min":920.58399999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.41599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0629999999999999,"5min":1.304,"15min":1.304},"last":0.84799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.222,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.222}]},{"osd":10,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.78500000000000003}]},{"osd":12,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":1.026},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.58899999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":25.651,"5min":25.651,"15min":25.651},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":501.07400000000001,"5min":501.07400000000001,"15min":501.07400000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":44.5,"5min":44.5,"15min":44.5},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":501.08800000000002,"5min":501.08800000000002,"15min":501.08800000000002},"last":1.2070000000000001}]},{"osd":15,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.403,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.96999999999999997}]}]},{"osd":13,"up_from":21,"seq":90194313245,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799440,"kb_used_data":92880240,"kb_used_omap":4484,"kb_used_meta":0,"kb_avail":92880176,"statfs":{"total":95927926784,"available":95109300224,"internally_reserved":65536,"allocated":95109365760,"data_stored":95109365760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4592085,"internal_metadata":0},"hb_peers":[2,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.1129999999999995,"5min":8.1129999999999995,"15min":8.1129999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":8.1430000000000007,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":0.60899999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":19.710000000000001,"5min":19.710000000000001,"15min":19.710000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":342.56099999999998,"5min":342.56099999999998,"15min":342.56099999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":28.227,"5min":28.227,"15min":28.227},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":495.40800000000002,"5min":495.40800000000002,"15min":495.40800000000002},"last":0.60099999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.7960000000000003,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.7619999999999996,"5min":5.7619999999999996,"15min":5.7619999999999996},"last":0.64600000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.373}]},{"osd":6,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":18.945,"5min":18.945,"15min":18.945},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":293.34800000000001,"5min":293.34800000000001,"15min":293.34800000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.40100000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.4779999999999998,"5min":6.4779999999999998,"15min":6.4779999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":5.726,"5min":5.726,"15min":5.726},"last":0.73199999999999998}]},{"osd":8,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":44.985999999999997,"5min":44.985999999999997,"15min":44.985999999999997},"last":0.52200000000000002}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.63100000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54000000000000004}]},{"osd":14,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.70899999999999996}]},{"osd":15,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.746}]}]},{"osd":2,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798928,"kb_used_data":92880748,"kb_used_omap":4392,"kb_used_meta":0,"kb_avail":92880688,"statfs":{"total":95927926784,"available":95109824512,"internally_reserved":61440,"allocated":95109885952,"data_stored":95109885952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4498429,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95199999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86599999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.38400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":46.487000000000002,"5min":9.6280000000000001,"15min":3.4849999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":829.03399999999999,"5min":829.03399999999999,"15min":829.03399999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.43099999999999999,"15min":0.41099999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.72899999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":39.168999999999997,"5min":13.167999999999999,"15min":8.8339999999999996},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":655.90899999999999,"5min":655.90899999999999,"15min":655.90899999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61299999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0329999999999999,"5min":1.329,"15min":1.329},"last":0.79700000000000004},{"interface":"front","average":{"1min":54.841000000000001,"5min":11.481,"15min":4.2539999999999996},"min":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":976.00800000000004,"5min":976.00800000000004,"15min":976.00800000000004},"last":0.74399999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63}]},{"osd":11,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.68700000000000006,"15min":0.63600000000000001},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":6.4240000000000004,"5min":6.4240000000000004,"15min":6.4240000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.74399999999999999,"15min":0.68999999999999995},"min":{"1min":0.47299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":6.4390000000000001,"5min":6.4390000000000001,"15min":6.4390000000000001},"last":0.89800000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":47.031999999999996,"5min":9.9469999999999992,"15min":3.766},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":834.505,"5min":834.505,"15min":834.505},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97199999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.71599999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99199999999999999,"5min":1.363,"15min":1.363},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.88}]},{"osd":14,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.92200000000000004}]}]},{"osd":15,"up_from":21,"seq":90194313246,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798156,"kb_used_data":92881472,"kb_used_omap":3194,"kb_used_meta":0,"kb_avail":92881460,"statfs":{"total":95927926784,"available":95110615040,"internally_reserved":12288,"allocated":95110627328,"data_stored":95110627328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3271314,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.79300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":12.098000000000001,"5min":2.8300000000000001,"15min":1.2849999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":162.16800000000001,"5min":162.16800000000001,"15min":162.16800000000001},"last":0.76500000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":47.701000000000001,"5min":9.8719999999999999,"15min":3.5670000000000002},"min":{"1min":0.26600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":662.84000000000003,"5min":662.84000000000003,"15min":662.84000000000003},"last":0.39700000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64200000000000002,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57099999999999995,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.40400000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94899999999999995,"5min":1.385,"15min":1.385},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.85699999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":4.798,"15min":5.5069999999999997},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69999999999999996,"5min":90.775999999999996,"15min":90.775999999999996},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Sep 15 01:33:36 2024","interfaces":[{"interface":"back","average":{"1min":50.158999999999999,"5min":50.158999999999999,"15min":50.158999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":894.27700000000004,"5min":894.27700000000004,"15min":894.27700000000004},"last":0.497},{"interface":"front","average":{"1min":49.308,"5min":49.308,"15min":49.308},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":878.12199999999996,"5min":878.12199999999996,"15min":878.12199999999996},"last":0.61299999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":50.171999999999997,"5min":10.541,"15min":3.9359999999999999},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":645.34900000000005,"5min":645.34900000000005,"15min":645.34900000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":27.992000000000001,"15min":32.540999999999997},"min":{"1min":0.45900000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99099999999999999,"5min":542.06600000000003,"15min":542.06600000000003},"last":0.59699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":1.603,"15min":1.603},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96199999999999997,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.78200000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":1.163,"15min":1.163},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75,"5min":12.249000000000001,"15min":14.164999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.224,"5min":246.56100000000001,"15min":246.56100000000001},"last":0.81599999999999995}]}]},{"osd":4,"up_from":21,"seq":90194313245,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798196,"kb_used_data":92881444,"kb_used_omap":3058,"kb_used_meta":0,"kb_avail":92881420,"statfs":{"total":95927926784,"available":95110574080,"internally_reserved":24576,"allocated":95110598656,"data_stored":95110598656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3131934,"internal_metadata":0},"hb_peers":[0,3,5,6,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":17.896999999999998,"5min":17.896999999999998,"15min":17.896999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":314.661,"5min":314.661,"15min":314.661},"last":0.42499999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51300000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.747},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":1.127}]},{"osd":3,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.55900000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0940000000000001,"5min":1.179,"15min":1.179},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.45000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.622},{"interface":"front","average":{"1min":42.378,"5min":24.170000000000002,"15min":21.135000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":918.75699999999995,"5min":918.75699999999995,"15min":918.75699999999995},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":7.5110000000000001,"5min":2.1309999999999998,"15min":1.2350000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":142.26599999999999,"5min":142.26599999999999,"15min":142.26599999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.57899999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1719999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0840000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.871}]},{"osd":9,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78300000000000003,"15min":0.77900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.59499999999999997}]},{"osd":10,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":40.198999999999998,"5min":8.7089999999999996,"15min":3.4609999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":828.54499999999996,"5min":828.54499999999996,"15min":828.54499999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":14.634,"5min":3.5920000000000001,"15min":1.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":291.08800000000002,"5min":291.08800000000002,"15min":291.08800000000002},"last":0.67800000000000005}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":28.721,"5min":28.721,"15min":28.721},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":558.56700000000001,"5min":558.56700000000001,"15min":558.56700000000001},"last":0.57999999999999996}]},{"osd":12,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1799999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.5}]},{"osd":13,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.71699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.61099999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3169999999999999,"5min":1.325,"15min":1.325},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.90500000000000003,"15min":0.90700000000000003},"min":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.256,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.64300000000000002}]}]},{"osd":5,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798296,"kb_used_data":92881368,"kb_used_omap":3055,"kb_used_meta":0,"kb_avail":92881320,"statfs":{"total":95927926784,"available":95110471680,"internally_reserved":49152,"allocated":95110520832,"data_stored":95110520832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3128858,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.499},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.70799999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.88300000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72599999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.628,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.48,"5min":1.524,"15min":1.524},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.70199999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.80800000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92200000000000004,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.60599999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":36.902999999999999,"5min":7.8339999999999996,"15min":2.9889999999999999},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":764.33399999999995,"5min":764.33399999999995,"15min":764.33399999999995},"last":0.441},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.39200000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1899999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":17.335999999999999,"5min":22.786000000000001,"15min":23.693999999999999},"min":{"1min":0.496,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":346.49200000000002,"5min":492.697,"15min":492.697},"last":0.78800000000000003}]},{"osd":15,"last update":"Sun Sep 15 01:33:26 2024","interfaces":[{"interface":"back","average":{"1min":13.939,"5min":13.939,"15min":13.939},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":291.12599999999998,"5min":291.12599999999998,"15min":291.12599999999998},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.93700000000000006}]}]},{"osd":6,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798300,"kb_used_data":92881364,"kb_used_omap":3133,"kb_used_meta":0,"kb_avail":92881316,"statfs":{"total":95927926784,"available":95110467584,"internally_reserved":49152,"allocated":95110516736,"data_stored":95110516736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3208697,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.85699999999999998,"15min":0.88100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.165,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77600000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.878,"5min":1.264,"15min":1.264},"last":0.60499999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.53800000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77800000000000002,"5min":1.163,"15min":1.163},"last":0.755}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.379,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83799999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.746,"15min":0.746},"last":0.49399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.90200000000000002}]},{"osd":10,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":61.164000000000001,"5min":12.723000000000001,"15min":4.649},"min":{"1min":0.48099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":656.21000000000004,"5min":656.21000000000004,"15min":656.21000000000004},"last":0.83399999999999996}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":11.742000000000001,"5min":2.8580000000000001,"15min":1.3779999999999999},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":200.52799999999999,"5min":200.52799999999999,"15min":200.52799999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.64600000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82999999999999996,"5min":1.353,"15min":1.353},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.99199999999999999,"15min":1.036},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97999999999999998,"5min":5.8470000000000004,"15min":5.8470000000000004},"last":0.85799999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.55800000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.55200000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.034,"5min":1.212,"15min":1.212},"last":0.91500000000000004}]}]},{"osd":7,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881452,"kb_used_omap":2577,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":61440,"allocated":95110606848,"data_stored":95110606848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2639163,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.48199999999999998,"15min":0.47099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":47.609999999999999,"15min":55.453000000000003},"min":{"1min":0.36699999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.69999999999999996,"5min":874.39499999999998,"15min":874.39499999999998},"last":0.38300000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":4.7350000000000003,"5min":4.7350000000000003,"15min":4.7350000000000003},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.33700000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.58299999999999996}]},{"osd":6,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":1.165,"15min":1.165},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":1.0880000000000001,"15min":1.167},"min":{"1min":0.42799999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":5.6539999999999999,"15min":5.6539999999999999},"last":0.52400000000000002}]},{"osd":8,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55900000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":4.4359999999999999,"15min":5.069},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84899999999999998,"5min":62.991999999999997,"15min":62.991999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.50700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67600000000000005,"15min":0.68400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":1.21,"15min":1.21},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":43.418999999999997,"15min":50.549999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93400000000000005,"5min":792.73599999999999,"15min":792.73599999999999},"last":0.47299999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.091,"15min":1.157},"min":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92100000000000004,"5min":8.1489999999999991,"15min":8.1489999999999991},"last":0.68999999999999995}]},{"osd":14,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.47199999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.878,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":41.905000000000001,"15min":48.776000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.83099999999999996,"5min":773.18799999999999,"15min":773.18799999999999},"last":0.72699999999999998}]},{"osd":15,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.1220000000000001,"15min":1.216},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79000000000000004,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83399999999999996,"5min":1.018,"15min":1.018},"last":0.60599999999999998}]}]},{"osd":8,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798352,"kb_used_data":92881324,"kb_used_omap":3898,"kb_used_meta":0,"kb_avail":92881264,"statfs":{"total":95927926784,"available":95110414336,"internally_reserved":61440,"allocated":95110475776,"data_stored":95110475776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3991961,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.373},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.626,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.40500000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.503},{"interface":"front","average":{"1min":24.794,"5min":24.794,"15min":24.794},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":413.66800000000001,"5min":413.66800000000001,"15min":413.66800000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55300000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.44800000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72499999999999998,"5min":1.093,"15min":1.093},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.44700000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.95599999999999996,"5min":1.073,"15min":1.073},"last":0.57499999999999996}]},{"osd":7,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":30.347000000000001,"5min":6.5869999999999997,"15min":2.6269999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":505.29300000000001,"5min":505.29300000000001,"15min":505.29300000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":12.156000000000001,"5min":2.9510000000000001,"15min":1.417},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":196.00299999999999,"5min":196.00299999999999,"15min":196.00299999999999},"last":0.79900000000000004}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.46899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97699999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.75600000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0249999999999999,"5min":1.208,"15min":1.208},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.004,"5min":1.351,"15min":1.351},"last":0.84799999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.503,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66100000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":1.054,"15min":1.054},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98999999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.67500000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798172,"kb_used_data":92881492,"kb_used_omap":3280,"kb_used_meta":0,"kb_avail":92881444,"statfs":{"total":95927926784,"available":95110598656,"internally_reserved":49152,"allocated":95110647808,"data_stored":95110647808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3358734,"internal_metadata":0},"hb_peers":[3,5,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.899},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75700000000000001,"15min":0.752},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":3.9750000000000001,"5min":3.9750000000000001,"15min":3.9750000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":3.9039999999999999,"5min":4.3150000000000004,"15min":4.383},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":56.188000000000002,"5min":64.938999999999993,"15min":64.938999999999993},"last":0.84799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":4.7960000000000003,"5min":1.5009999999999999,"15min":0.95199999999999996},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":75.778999999999996,"5min":75.778999999999996,"15min":75.778999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.75900000000000001,"15min":0.78100000000000003},"min":{"1min":0.45300000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84599999999999997,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.47999999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":44.508000000000003,"15min":51.82},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96299999999999997,"5min":500.92099999999999,"15min":500.92099999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.41399999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.83699999999999997}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.52500000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84199999999999997,"15min":0.87},"min":{"1min":0.503,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0229999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.82299999999999995}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.4460000000000002},{"interface":"front","average":{"1min":36.435000000000002,"5min":36.435000000000002,"15min":36.435000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":641.649,"5min":641.649,"15min":641.649},"last":2.4740000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.048},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.92800000000000005}]},{"osd":15,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.93899999999999995,"15min":0.97299999999999998},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.036,"5min":6.0670000000000002,"15min":6.0670000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0089999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.94799999999999995}]}]},{"osd":11,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798032,"kb_used_data":92881596,"kb_used_omap":3335,"kb_used_meta":0,"kb_avail":92881584,"statfs":{"total":95927926784,"available":95110742016,"internally_reserved":12288,"allocated":95110754304,"data_stored":95110754304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3415222,"internal_metadata":0},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":46.469000000000001,"5min":46.469000000000001,"15min":46.469000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":873.95500000000004,"5min":873.95500000000004,"15min":873.95500000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.954,"5min":1.954,"15min":1.954},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":28.431999999999999,"5min":28.431999999999999,"15min":28.431999999999999},"last":0.50900000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":565.90700000000004}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":13.093,"5min":3.125,"15min":1.4630000000000001},"min":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":250.767,"5min":250.767,"15min":250.767},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84899999999999998,"5min":0.871,"15min":0.871},"last":0.495}]},{"osd":5,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.623}]},{"osd":6,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.30199999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.433},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47699999999999998}]},{"osd":9,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51300000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86899999999999999,"5min":1.042,"15min":1.042},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90200000000000002,"5min":2.262,"15min":2.262},"last":0.39500000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.79000000000000004,"15min":0.81799999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93400000000000005,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.82099999999999995,"15min":0.84699999999999998},"min":{"1min":0.45300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85999999999999999,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":0.58599999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":1.903,"5min":0.95599999999999996,"15min":0.79800000000000004},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":25.056999999999999,"5min":25.056999999999999,"15min":25.056999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.442,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.54200000000000004}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":4.4009999999999998,"5min":13.781000000000001,"15min":15.343999999999999},"min":{"1min":0.51200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":74.078000000000003,"5min":310.78199999999998,"15min":310.78199999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.436,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97099999999999997,"5min":2.238,"15min":2.238},"last":0.52400000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":12,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798284,"kb_used_data":92881380,"kb_used_omap":1469,"kb_used_meta":0,"kb_avail":92881332,"statfs":{"total":95927926784,"available":95110483968,"internally_reserved":49152,"allocated":95110533120,"data_stored":95110533120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1505179,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.28000000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":8.0050000000000008,"5min":8.0050000000000008,"15min":8.0050000000000008},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":0.35599999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.47099999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.51500000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":24.937000000000001,"5min":24.937000000000001,"15min":24.937000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":488.69,"5min":488.69,"15min":488.69},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5419999999999998,"5min":3.5419999999999998,"15min":3.5419999999999998},"last":0.29599999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.6309999999999998,"5min":3.6309999999999998,"15min":3.6309999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.57899999999999996}]},{"osd":11,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.499}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.433},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.40400000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.54300000000000004}]},{"osd":15,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.56899999999999995}]}]},{"osd":1,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798292,"kb_used_data":92881396,"kb_used_omap":3216,"kb_used_meta":0,"kb_avail":92881324,"statfs":{"total":95927926784,"available":95110475776,"internally_reserved":73728,"allocated":95110549504,"data_stored":95110549504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3294094,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":55.167999999999999,"5min":11.429,"15min":4.1390000000000002},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.31},"max":{"1min":875.06100000000004,"5min":875.06100000000004,"15min":875.06100000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.754,"15min":0.754},"last":0.504}]},{"osd":2,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79100000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61299999999999999,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91100000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59099999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.79300000000000004}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.29999999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79200000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":28.789000000000001,"15min":33.491},"min":{"1min":0.38400000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82999999999999996,"5min":635.46400000000006,"15min":635.46400000000006},"last":0.41999999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.53800000000000003},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71199999999999997,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.80500000000000005,"15min":0.85699999999999998},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72699999999999998,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.69799999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":54.337000000000003,"5min":11.387,"15min":4.2290000000000001},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":806.31200000000001,"5min":806.31200000000001,"15min":806.31200000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68000000000000005,"15min":0.68600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83799999999999997,"5min":1.341,"15min":1.341},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.75,"15min":0.751},"min":{"1min":0.52600000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.4379999999999999,"5min":1.825,"15min":1.825},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.48499999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0469999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.82499999999999996}]},{"osd":8,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":4.0830000000000002,"5min":1.395,"15min":0.94699999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":55.694000000000003,"5min":55.694000000000003,"15min":55.694000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69199999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.88900000000000001,"5min":1.492,"15min":1.492},"last":0.65500000000000003}]},{"osd":10,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":2.149,"15min":2.222},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":17.527000000000001,"5min":27.920000000000002,"15min":27.920000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":8.1069999999999993,"15min":9.3320000000000007},"min":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.026,"5min":166.16200000000001,"15min":166.16200000000001},"last":0.80800000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.755},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.496,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84999999999999998,"5min":1.393,"15min":1.393},"last":0.60999999999999999}]}]}],"pool_statfs":[]}} 2024-09-15T01:34:26.527 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T01:34:26.527 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-15T01:34:25.095499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":5,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":12773700,"kb_used_data":1486100956,"kb_used_omap":52112,"kb_used_meta":0,"kb_avail":1486100156,"statfs":{"total":1534846828544,"available":1521766559744,"internally_reserved":819200,"allocated":1521767378944,"data_stored":1521767378944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53363571,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001207"},"pg_stats":[{"pgid":"1.0","version":"106'114","reported_seq":266,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888285+0000","last_change":"2024-09-15T01:33:49.902092+0000","last_active":"2024-09-15T01:34:19.888285+0000","last_peered":"2024-09-15T01:34:19.888285+0000","last_clean":"2024-09-15T01:34:19.888285+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:34:19.888285+0000","last_undegraded":"2024-09-15T01:34:19.888285+0000","last_fullsized":"2024-09-15T01:34:19.888285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'114","last_scrub_stamp":"2024-09-15T01:33:49.902036+0000","last_deep_scrub":"106'114","last_deep_scrub_stamp":"2024-09-15T01:33:49.902036+0000","last_clean_scrub_stamp":"2024-09-15T01:33:49.902036+0000","objects_scrubbed":2,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:12.193331+0000","scrub_duration":0.010874557,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"105'2","reported_seq":128,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.341551+0000","last_change":"2024-09-15T01:33:48.491848+0000","last_active":"2024-09-15T01:34:19.341551+0000","last_peered":"2024-09-15T01:34:19.341551+0000","last_clean":"2024-09-15T01:34:19.341551+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:34:19.341551+0000","last_undegraded":"2024-09-15T01:34:19.341551+0000","last_fullsized":"2024-09-15T01:34:19.341551+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'2","last_scrub_stamp":"2024-09-15T01:33:48.491771+0000","last_deep_scrub":"105'2","last_deep_scrub_stamp":"2024-09-15T01:33:48.491771+0000","last_clean_scrub_stamp":"2024-09-15T01:33:48.491771+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:01.566416+0000","scrub_duration":0.0025105769999999999,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.794589+0000","last_change":"2024-09-15T01:33:46.379860+0000","last_active":"2024-09-15T01:34:20.794589+0000","last_peered":"2024-09-15T01:34:20.794589+0000","last_clean":"2024-09-15T01:34:20.794589+0000","last_became_active":"2024-09-15T01:33:23.083242+0000","last_became_peered":"2024-09-15T01:33:23.083242+0000","last_unstale":"2024-09-15T01:34:20.794589+0000","last_undegraded":"2024-09-15T01:34:20.794589+0000","last_fullsized":"2024-09-15T01:34:20.794589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:46.379809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:46.379809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:49.087667+0000","scrub_duration":0.001946382,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8],"acting":[10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888248+0000","last_change":"2024-09-15T01:33:50.941845+0000","last_active":"2024-09-15T01:34:19.888248+0000","last_peered":"2024-09-15T01:34:19.888248+0000","last_clean":"2024-09-15T01:34:19.888248+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:34:19.888248+0000","last_undegraded":"2024-09-15T01:34:19.888248+0000","last_fullsized":"2024-09-15T01:34:19.888248+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:50.941790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:50.941790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:10.537027+0000","scrub_duration":0.0017165430000000001,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":133,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.484672+0000","last_change":"2024-09-15T01:33:46.870657+0000","last_active":"2024-09-15T01:34:19.484672+0000","last_peered":"2024-09-15T01:34:19.484672+0000","last_clean":"2024-09-15T01:34:19.484672+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:34:19.484672+0000","last_undegraded":"2024-09-15T01:34:19.484672+0000","last_fullsized":"2024-09-15T01:34:19.484672+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-15T01:33:46.870584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:46.870584+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:14.306792+0000","scrub_duration":0.0027185820000000002,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888035+0000","last_change":"2024-09-15T01:33:41.548849+0000","last_active":"2024-09-15T01:34:19.888035+0000","last_peered":"2024-09-15T01:34:19.888035+0000","last_clean":"2024-09-15T01:34:19.888035+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:34:19.888035+0000","last_undegraded":"2024-09-15T01:34:19.888035+0000","last_fullsized":"2024-09-15T01:34:19.888035+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:41.548799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:41.548799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:55.947522+0000","scrub_duration":0.001258681,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:19.888166+0000","last_change":"2024-09-15T01:33:41.543896+0000","last_active":"2024-09-15T01:34:19.888166+0000","last_peered":"2024-09-15T01:34:19.888166+0000","last_clean":"2024-09-15T01:34:19.888166+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:34:19.888166+0000","last_undegraded":"2024-09-15T01:34:19.888166+0000","last_fullsized":"2024-09-15T01:34:19.888166+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:41.543850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:41.543850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:56.945346+0000","scrub_duration":0.0015955419999999999,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.587067+0000","last_change":"2024-09-15T01:33:39.638452+0000","last_active":"2024-09-15T01:34:20.587067+0000","last_peered":"2024-09-15T01:34:20.587067+0000","last_clean":"2024-09-15T01:34:20.587067+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:34:20.587067+0000","last_undegraded":"2024-09-15T01:34:20.587067+0000","last_fullsized":"2024-09-15T01:34:20.587067+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:39.638404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:39.638404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:59.511423+0000","scrub_duration":0.001362835,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":129,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T01:34:20.794259+0000","last_change":"2024-09-15T01:33:38.175676+0000","last_active":"2024-09-15T01:34:20.794259+0000","last_peered":"2024-09-15T01:34:20.794259+0000","last_clean":"2024-09-15T01:34:20.794259+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:34:20.794259+0000","last_undegraded":"2024-09-15T01:34:20.794259+0000","last_fullsized":"2024-09-15T01:34:20.794259+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:38.175625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:38.175625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:44.144155+0000","scrub_duration":0.001339526,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881456,"kb_used_omap":3954,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":65536,"allocated":95110610944,"data_stored":95110610944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4049435,"internal_metadata":0},"hb_peers":[1,2,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":16.196999999999999,"5min":27.753,"15min":29.678999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":299.721,"5min":574.26900000000001,"15min":574.26900000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":18.404,"5min":4.0270000000000001,"15min":1.631},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":342.08300000000003,"5min":342.08300000000003,"15min":342.08300000000003},"last":0.35799999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:33:20 2024","interfaces":[{"interface":"back","average":{"1min":32.810000000000002,"5min":32.810000000000002,"15min":32.810000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":644.97699999999998,"5min":644.97699999999998,"15min":644.97699999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":805.39499999999998}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.71399999999999997,"15min":0.73999999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.69299999999999995,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":11.887,"5min":2.8730000000000002,"15min":1.371},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":214.994,"5min":214.994,"15min":214.994},"last":0.66900000000000004}]},{"osd":5,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47699999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.502}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82899999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.621},{"interface":"front","average":{"1min":10.272,"5min":2.625,"15min":1.351},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":173.11099999999999,"5min":173.11099999999999,"15min":173.11099999999999},"last":0.48599999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":6.056,"15min":6.9420000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.589,"5min":134.12200000000001,"15min":134.12200000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":45.521000000000001,"5min":9.7249999999999996,"15min":3.7589999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":846.97400000000005,"5min":846.97400000000005,"15min":846.97400000000005},"last":0.65400000000000003}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.56599999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.5649999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":17.137,"5min":4.0940000000000003,"15min":1.9199999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":313.495,"5min":313.495,"15min":313.495},"last":0.72499999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.69399999999999995}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81399999999999995,"15min":0.81299999999999994},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.70899999999999996}]}]},{"osd":9,"up_from":21,"seq":90194313244,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881452,"kb_used_omap":1548,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":61440,"allocated":95110606848,"data_stored":95110606848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1586173,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.46999999999999997},"min":{"1min":0.38700000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.49299999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":23.486999999999998,"5min":5.2149999999999999,"15min":2.1699999999999999},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":462.93700000000001,"5min":462.93700000000001,"15min":462.93700000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.94399999999999995,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84399999999999997,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89800000000000002,"5min":1.97,"15min":1.97},"last":0.93100000000000005}]},{"osd":8,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.0720000000000001,"15min":1.1519999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":10.916,"15min":10.916},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.623,"5min":1.218,"15min":1.3169999999999999},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":13.147,"15min":13.147},"last":0.59499999999999997}]},{"osd":10,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":5.8819999999999997,"5min":1.702,"15min":1.006},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":127.033,"5min":127.033,"15min":127.033},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.90600000000000003,"15min":0.95299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92900000000000005,"5min":6.4539999999999997,"15min":6.4539999999999997},"last":0.877}]},{"osd":11,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":10.221,"5min":2.5840000000000001,"15min":1.3109999999999999},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":229.28899999999999,"5min":229.28899999999999,"15min":229.28899999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.88700000000000001,"15min":0.93100000000000005},"min":{"1min":0.42199999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.874,"5min":6.9489999999999998,"15min":6.9489999999999998},"last":0.68600000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":23.875,"15min":27.75},"min":{"1min":0.47899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97699999999999998,"5min":495.95800000000003,"15min":495.95800000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":43.860999999999997,"15min":51.058999999999997},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":914.18499999999995,"15min":914.18499999999995},"last":0.94999999999999996}]},{"osd":13,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":2.7599999999999998,"5min":1.595,"15min":1.401},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":49.841000000000001,"5min":49.841000000000001,"15min":49.841000000000001},"last":0.84699999999999998}]},{"osd":14,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86099999999999999,"5min":1.96,"15min":1.96},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.96199999999999997}]},{"osd":15,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98799999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91000000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.66300000000000003}]}]},{"osd":3,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798228,"kb_used_data":92881448,"kb_used_omap":2127,"kb_used_meta":0,"kb_avail":92881388,"statfs":{"total":95927926784,"available":95110541312,"internally_reserved":61440,"allocated":95110602752,"data_stored":95110602752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2179043,"internal_metadata":0},"hb_peers":[0,2,4,6,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.365,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.91700000000000004}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.57699999999999996,"5min":0.312,"15min":0.312},"max":{"1min":0.99099999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.503}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.81399999999999995,"15min":0.84699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1850000000000001,"5min":7.4720000000000004,"15min":7.4720000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":7.5629999999999997,"15min":8.7230000000000008},"min":{"1min":0.41099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.1100000000000001,"5min":182.87,"15min":182.87},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.5,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":3.202},{"interface":"front","average":{"1min":28.616,"5min":6.2119999999999997,"15min":2.4780000000000002},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":501.65800000000002,"5min":501.65800000000002,"15min":501.65800000000002},"last":0.56200000000000006}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":24.288,"5min":24.288,"15min":24.288},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":500.678,"5min":500.678,"15min":500.678},"last":0.55100000000000005},{"interface":"front","average":{"1min":35.298999999999999,"5min":35.298999999999999,"15min":35.298999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":500.79899999999998,"5min":500.79899999999998,"15min":500.79899999999998},"last":0.52200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":11.316000000000001,"5min":11.316000000000001,"15min":11.316000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":200.40799999999999,"5min":200.40799999999999,"15min":200.40799999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.59799999999999998}]},{"osd":11,"last update":"Sun Sep 15 01:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.433}]},{"osd":12,"last update":"Sun Sep 15 01:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.61199999999999999}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.78400000000000003,"15min":0.76800000000000002},"min":{"1min":0.67900000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.427,"5min":1.637,"15min":1.637},"last":0.624},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0169999999999999,"15min":1.0389999999999999},"min":{"1min":0.63200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.476,"5min":7.4500000000000002,"15min":7.4500000000000002},"last":0.57999999999999996}]},{"osd":14,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53200000000000003}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.52200000000000002}]}]},{"osd":0,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798352,"kb_used_data":92881324,"kb_used_omap":5384,"kb_used_meta":0,"kb_avail":92881264,"statfs":{"total":95927926784,"available":95110414336,"internally_reserved":61440,"allocated":95110475776,"data_stored":95110475776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5513250,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.872},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.113,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.113}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":31.643999999999998,"15min":36.808999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.488,"5min":814.68299999999999,"15min":814.68299999999999},"last":0.624},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.63700000000000001,"15min":0.65200000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1890000000000001,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":40.415999999999997,"5min":8.5380000000000003,"15min":3.2250000000000001},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":799.04999999999995,"5min":799.04999999999995,"15min":799.04999999999995},"last":0.66500000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69399999999999995,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.2769999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.264,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.133}]},{"osd":7,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":46.667999999999999,"5min":9.8330000000000002,"15min":3.694},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":920.58399999999995,"5min":920.58399999999995,"15min":920.58399999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.41599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0629999999999999,"5min":1.304,"15min":1.304},"last":0.84799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.222,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.222}]},{"osd":10,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.78500000000000003}]},{"osd":12,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":1.026},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.58899999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":25.651,"5min":25.651,"15min":25.651},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":501.07400000000001,"5min":501.07400000000001,"15min":501.07400000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":44.5,"5min":44.5,"15min":44.5},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":501.08800000000002,"5min":501.08800000000002,"15min":501.08800000000002},"last":1.2070000000000001}]},{"osd":15,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.403,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.96999999999999997}]}]},{"osd":13,"up_from":21,"seq":90194313245,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799440,"kb_used_data":92880240,"kb_used_omap":4484,"kb_used_meta":0,"kb_avail":92880176,"statfs":{"total":95927926784,"available":95109300224,"internally_reserved":65536,"allocated":95109365760,"data_stored":95109365760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4592085,"internal_metadata":0},"hb_peers":[2,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.1129999999999995,"5min":8.1129999999999995,"15min":8.1129999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":8.1430000000000007,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":0.60899999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":19.710000000000001,"5min":19.710000000000001,"15min":19.710000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":342.56099999999998,"5min":342.56099999999998,"15min":342.56099999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":28.227,"5min":28.227,"15min":28.227},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":495.40800000000002,"5min":495.40800000000002,"15min":495.40800000000002},"last":0.60099999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.7960000000000003,"5min":5.7960000000000003,"15min":5.7960000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.7619999999999996,"5min":5.7619999999999996,"15min":5.7619999999999996},"last":0.64600000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.373}]},{"osd":6,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":18.945,"5min":18.945,"15min":18.945},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":293.34800000000001,"5min":293.34800000000001,"15min":293.34800000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.40100000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.4779999999999998,"5min":6.4779999999999998,"15min":6.4779999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":5.726,"5min":5.726,"15min":5.726},"last":0.73199999999999998}]},{"osd":8,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":44.985999999999997,"5min":44.985999999999997,"15min":44.985999999999997},"last":0.52200000000000002}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.63100000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54000000000000004}]},{"osd":14,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.70899999999999996}]},{"osd":15,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.746}]}]},{"osd":2,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798928,"kb_used_data":92880748,"kb_used_omap":4392,"kb_used_meta":0,"kb_avail":92880688,"statfs":{"total":95927926784,"available":95109824512,"internally_reserved":61440,"allocated":95109885952,"data_stored":95109885952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4498429,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95199999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86599999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.38400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":46.487000000000002,"5min":9.6280000000000001,"15min":3.4849999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":829.03399999999999,"5min":829.03399999999999,"15min":829.03399999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.43099999999999999,"15min":0.41099999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.72899999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":39.168999999999997,"5min":13.167999999999999,"15min":8.8339999999999996},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":655.90899999999999,"5min":655.90899999999999,"15min":655.90899999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61299999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0329999999999999,"5min":1.329,"15min":1.329},"last":0.79700000000000004},{"interface":"front","average":{"1min":54.841000000000001,"5min":11.481,"15min":4.2539999999999996},"min":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":976.00800000000004,"5min":976.00800000000004,"15min":976.00800000000004},"last":0.74399999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63}]},{"osd":11,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.68700000000000006,"15min":0.63600000000000001},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":6.4240000000000004,"5min":6.4240000000000004,"15min":6.4240000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.74399999999999999,"15min":0.68999999999999995},"min":{"1min":0.47299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":6.4390000000000001,"5min":6.4390000000000001,"15min":6.4390000000000001},"last":0.89800000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":47.031999999999996,"5min":9.9469999999999992,"15min":3.766},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":834.505,"5min":834.505,"15min":834.505},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97199999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.71599999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99199999999999999,"5min":1.363,"15min":1.363},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.88}]},{"osd":14,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.92200000000000004}]}]},{"osd":15,"up_from":21,"seq":90194313246,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798156,"kb_used_data":92881472,"kb_used_omap":3194,"kb_used_meta":0,"kb_avail":92881460,"statfs":{"total":95927926784,"available":95110615040,"internally_reserved":12288,"allocated":95110627328,"data_stored":95110627328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3271314,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.79300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":12.098000000000001,"5min":2.8300000000000001,"15min":1.2849999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":162.16800000000001,"5min":162.16800000000001,"15min":162.16800000000001},"last":0.76500000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":47.701000000000001,"5min":9.8719999999999999,"15min":3.5670000000000002},"min":{"1min":0.26600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":662.84000000000003,"5min":662.84000000000003,"15min":662.84000000000003},"last":0.39700000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64200000000000002,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57099999999999995,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.40400000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94899999999999995,"5min":1.385,"15min":1.385},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.85699999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":4.798,"15min":5.5069999999999997},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69999999999999996,"5min":90.775999999999996,"15min":90.775999999999996},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Sep 15 01:33:36 2024","interfaces":[{"interface":"back","average":{"1min":50.158999999999999,"5min":50.158999999999999,"15min":50.158999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":894.27700000000004,"5min":894.27700000000004,"15min":894.27700000000004},"last":0.497},{"interface":"front","average":{"1min":49.308,"5min":49.308,"15min":49.308},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":878.12199999999996,"5min":878.12199999999996,"15min":878.12199999999996},"last":0.61299999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":50.171999999999997,"5min":10.541,"15min":3.9359999999999999},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":645.34900000000005,"5min":645.34900000000005,"15min":645.34900000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":27.992000000000001,"15min":32.540999999999997},"min":{"1min":0.45900000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99099999999999999,"5min":542.06600000000003,"15min":542.06600000000003},"last":0.59699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":1.603,"15min":1.603},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96199999999999997,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.78200000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":1.163,"15min":1.163},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75,"5min":12.249000000000001,"15min":14.164999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.224,"5min":246.56100000000001,"15min":246.56100000000001},"last":0.81599999999999995}]}]},{"osd":4,"up_from":21,"seq":90194313245,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798196,"kb_used_data":92881444,"kb_used_omap":3058,"kb_used_meta":0,"kb_avail":92881420,"statfs":{"total":95927926784,"available":95110574080,"internally_reserved":24576,"allocated":95110598656,"data_stored":95110598656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3131934,"internal_metadata":0},"hb_peers":[0,3,5,6,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":17.896999999999998,"5min":17.896999999999998,"15min":17.896999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":314.661,"5min":314.661,"15min":314.661},"last":0.42499999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51300000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.747},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":1.127}]},{"osd":3,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.55900000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0940000000000001,"5min":1.179,"15min":1.179},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.45000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.622},{"interface":"front","average":{"1min":42.378,"5min":24.170000000000002,"15min":21.135000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":918.75699999999995,"5min":918.75699999999995,"15min":918.75699999999995},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":7.5110000000000001,"5min":2.1309999999999998,"15min":1.2350000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":142.26599999999999,"5min":142.26599999999999,"15min":142.26599999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.57899999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1719999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0840000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.871}]},{"osd":9,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78300000000000003,"15min":0.77900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.59499999999999997}]},{"osd":10,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":40.198999999999998,"5min":8.7089999999999996,"15min":3.4609999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":828.54499999999996,"5min":828.54499999999996,"15min":828.54499999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":14.634,"5min":3.5920000000000001,"15min":1.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":291.08800000000002,"5min":291.08800000000002,"15min":291.08800000000002},"last":0.67800000000000005}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":28.721,"5min":28.721,"15min":28.721},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":558.56700000000001,"5min":558.56700000000001,"15min":558.56700000000001},"last":0.57999999999999996}]},{"osd":12,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1799999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.5}]},{"osd":13,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.71699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.61099999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3169999999999999,"5min":1.325,"15min":1.325},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.90500000000000003,"15min":0.90700000000000003},"min":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.256,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.64300000000000002}]}]},{"osd":5,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798296,"kb_used_data":92881368,"kb_used_omap":3055,"kb_used_meta":0,"kb_avail":92881320,"statfs":{"total":95927926784,"available":95110471680,"internally_reserved":49152,"allocated":95110520832,"data_stored":95110520832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3128858,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.499},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.70799999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.88300000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72599999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.628,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.48,"5min":1.524,"15min":1.524},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.70199999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.80800000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92200000000000004,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.60599999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":36.902999999999999,"5min":7.8339999999999996,"15min":2.9889999999999999},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":764.33399999999995,"5min":764.33399999999995,"15min":764.33399999999995},"last":0.441},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.39200000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1899999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":17.335999999999999,"5min":22.786000000000001,"15min":23.693999999999999},"min":{"1min":0.496,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":346.49200000000002,"5min":492.697,"15min":492.697},"last":0.78800000000000003}]},{"osd":15,"last update":"Sun Sep 15 01:33:26 2024","interfaces":[{"interface":"back","average":{"1min":13.939,"5min":13.939,"15min":13.939},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":291.12599999999998,"5min":291.12599999999998,"15min":291.12599999999998},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.93700000000000006}]}]},{"osd":6,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798300,"kb_used_data":92881364,"kb_used_omap":3133,"kb_used_meta":0,"kb_avail":92881316,"statfs":{"total":95927926784,"available":95110467584,"internally_reserved":49152,"allocated":95110516736,"data_stored":95110516736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3208697,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.85699999999999998,"15min":0.88100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.165,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77600000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.878,"5min":1.264,"15min":1.264},"last":0.60499999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.53800000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77800000000000002,"5min":1.163,"15min":1.163},"last":0.755}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.379,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83799999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.746,"15min":0.746},"last":0.49399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.90200000000000002}]},{"osd":10,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":61.164000000000001,"5min":12.723000000000001,"15min":4.649},"min":{"1min":0.48099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":656.21000000000004,"5min":656.21000000000004,"15min":656.21000000000004},"last":0.83399999999999996}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":11.742000000000001,"5min":2.8580000000000001,"15min":1.3779999999999999},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":200.52799999999999,"5min":200.52799999999999,"15min":200.52799999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.64600000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82999999999999996,"5min":1.353,"15min":1.353},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.99199999999999999,"15min":1.036},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97999999999999998,"5min":5.8470000000000004,"15min":5.8470000000000004},"last":0.85799999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.55800000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.55200000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.034,"5min":1.212,"15min":1.212},"last":0.91500000000000004}]}]},{"osd":7,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798224,"kb_used_data":92881452,"kb_used_omap":2577,"kb_used_meta":0,"kb_avail":92881392,"statfs":{"total":95927926784,"available":95110545408,"internally_reserved":61440,"allocated":95110606848,"data_stored":95110606848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2639163,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.48199999999999998,"15min":0.47099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":47.609999999999999,"15min":55.453000000000003},"min":{"1min":0.36699999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.69999999999999996,"5min":874.39499999999998,"15min":874.39499999999998},"last":0.38300000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":4.7350000000000003,"5min":4.7350000000000003,"15min":4.7350000000000003},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.33700000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.58299999999999996}]},{"osd":6,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":1.165,"15min":1.165},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":1.0880000000000001,"15min":1.167},"min":{"1min":0.42799999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":5.6539999999999999,"15min":5.6539999999999999},"last":0.52400000000000002}]},{"osd":8,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55900000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":4.4359999999999999,"15min":5.069},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84899999999999998,"5min":62.991999999999997,"15min":62.991999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.50700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67600000000000005,"15min":0.68400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":1.21,"15min":1.21},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":43.418999999999997,"15min":50.549999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93400000000000005,"5min":792.73599999999999,"15min":792.73599999999999},"last":0.47299999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.091,"15min":1.157},"min":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92100000000000004,"5min":8.1489999999999991,"15min":8.1489999999999991},"last":0.68999999999999995}]},{"osd":14,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.47199999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.878,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":41.905000000000001,"15min":48.776000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.83099999999999996,"5min":773.18799999999999,"15min":773.18799999999999},"last":0.72699999999999998}]},{"osd":15,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.1220000000000001,"15min":1.216},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79000000000000004,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83399999999999996,"5min":1.018,"15min":1.018},"last":0.60599999999999998}]}]},{"osd":8,"up_from":21,"seq":90194313244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798352,"kb_used_data":92881324,"kb_used_omap":3898,"kb_used_meta":0,"kb_avail":92881264,"statfs":{"total":95927926784,"available":95110414336,"internally_reserved":61440,"allocated":95110475776,"data_stored":95110475776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3991961,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.373},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.626,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.40500000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.503},{"interface":"front","average":{"1min":24.794,"5min":24.794,"15min":24.794},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":413.66800000000001,"5min":413.66800000000001,"15min":413.66800000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55300000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.44800000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72499999999999998,"5min":1.093,"15min":1.093},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.44700000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.95599999999999996,"5min":1.073,"15min":1.073},"last":0.57499999999999996}]},{"osd":7,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":30.347000000000001,"5min":6.5869999999999997,"15min":2.6269999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":505.29300000000001,"5min":505.29300000000001,"15min":505.29300000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":12.156000000000001,"5min":2.9510000000000001,"15min":1.417},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":196.00299999999999,"5min":196.00299999999999,"15min":196.00299999999999},"last":0.79900000000000004}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.46899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97699999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.75600000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0249999999999999,"5min":1.208,"15min":1.208},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.004,"5min":1.351,"15min":1.351},"last":0.84799999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.503,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66100000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":1.054,"15min":1.054},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98999999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.67500000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798172,"kb_used_data":92881492,"kb_used_omap":3280,"kb_used_meta":0,"kb_avail":92881444,"statfs":{"total":95927926784,"available":95110598656,"internally_reserved":49152,"allocated":95110647808,"data_stored":95110647808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3358734,"internal_metadata":0},"hb_peers":[3,5,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.899},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75700000000000001,"15min":0.752},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":3.9750000000000001,"5min":3.9750000000000001,"15min":3.9750000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":3.9039999999999999,"5min":4.3150000000000004,"15min":4.383},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":56.188000000000002,"5min":64.938999999999993,"15min":64.938999999999993},"last":0.84799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":4.7960000000000003,"5min":1.5009999999999999,"15min":0.95199999999999996},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":75.778999999999996,"5min":75.778999999999996,"15min":75.778999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.75900000000000001,"15min":0.78100000000000003},"min":{"1min":0.45300000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84599999999999997,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.47999999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":44.508000000000003,"15min":51.82},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96299999999999997,"5min":500.92099999999999,"15min":500.92099999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.41399999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.83699999999999997}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.52500000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84199999999999997,"15min":0.87},"min":{"1min":0.503,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0229999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.82299999999999995}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.4460000000000002},{"interface":"front","average":{"1min":36.435000000000002,"5min":36.435000000000002,"15min":36.435000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":641.649,"5min":641.649,"15min":641.649},"last":2.4740000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.048},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.92800000000000005}]},{"osd":15,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.93899999999999995,"15min":0.97299999999999998},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.036,"5min":6.0670000000000002,"15min":6.0670000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0089999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.94799999999999995}]}]},{"osd":11,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798032,"kb_used_data":92881596,"kb_used_omap":3335,"kb_used_meta":0,"kb_avail":92881584,"statfs":{"total":95927926784,"available":95110742016,"internally_reserved":12288,"allocated":95110754304,"data_stored":95110754304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3415222,"internal_metadata":0},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":46.469000000000001,"5min":46.469000000000001,"15min":46.469000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":873.95500000000004,"5min":873.95500000000004,"15min":873.95500000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.954,"5min":1.954,"15min":1.954},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":28.431999999999999,"5min":28.431999999999999,"15min":28.431999999999999},"last":0.50900000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":565.90700000000004}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":13.093,"5min":3.125,"15min":1.4630000000000001},"min":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":250.767,"5min":250.767,"15min":250.767},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84899999999999998,"5min":0.871,"15min":0.871},"last":0.495}]},{"osd":5,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.623}]},{"osd":6,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.30199999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.433},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47699999999999998}]},{"osd":9,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51300000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86899999999999999,"5min":1.042,"15min":1.042},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90200000000000002,"5min":2.262,"15min":2.262},"last":0.39500000000000002}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.79000000000000004,"15min":0.81799999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93400000000000005,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.82099999999999995,"15min":0.84699999999999998},"min":{"1min":0.45300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85999999999999999,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":0.58599999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":1.903,"5min":0.95599999999999996,"15min":0.79800000000000004},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":25.056999999999999,"5min":25.056999999999999,"15min":25.056999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.442,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.54200000000000004}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":4.4009999999999998,"5min":13.781000000000001,"15min":15.343999999999999},"min":{"1min":0.51200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":74.078000000000003,"5min":310.78199999999998,"15min":310.78199999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.436,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97099999999999997,"5min":2.238,"15min":2.238},"last":0.52400000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":12,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798284,"kb_used_data":92881380,"kb_used_omap":1469,"kb_used_meta":0,"kb_avail":92881332,"statfs":{"total":95927926784,"available":95110483968,"internally_reserved":49152,"allocated":95110533120,"data_stored":95110533120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1505179,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.28000000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":8.0050000000000008,"5min":8.0050000000000008,"15min":8.0050000000000008},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":0.35599999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.47099999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.51500000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":24.937000000000001,"5min":24.937000000000001,"15min":24.937000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":488.69,"5min":488.69,"15min":488.69},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.5419999999999998,"5min":3.5419999999999998,"15min":3.5419999999999998},"last":0.29599999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.6309999999999998,"5min":3.6309999999999998,"15min":3.6309999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.57899999999999996}]},{"osd":11,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.499}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.433},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.40400000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.54300000000000004}]},{"osd":15,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.56899999999999995}]}]},{"osd":1,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798292,"kb_used_data":92881396,"kb_used_omap":3216,"kb_used_meta":0,"kb_avail":92881324,"statfs":{"total":95927926784,"available":95110475776,"internally_reserved":73728,"allocated":95110549504,"data_stored":95110549504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3294094,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":55.167999999999999,"5min":11.429,"15min":4.1390000000000002},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.31},"max":{"1min":875.06100000000004,"5min":875.06100000000004,"15min":875.06100000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.754,"15min":0.754},"last":0.504}]},{"osd":2,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79100000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61299999999999999,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91100000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59099999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.79300000000000004}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.29999999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79200000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":28.789000000000001,"15min":33.491},"min":{"1min":0.38400000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82999999999999996,"5min":635.46400000000006,"15min":635.46400000000006},"last":0.41999999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.53800000000000003},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71199999999999997,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.80500000000000005,"15min":0.85699999999999998},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72699999999999998,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.69799999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":54.337000000000003,"5min":11.387,"15min":4.2290000000000001},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":806.31200000000001,"5min":806.31200000000001,"15min":806.31200000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68000000000000005,"15min":0.68600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83799999999999997,"5min":1.341,"15min":1.341},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.75,"15min":0.751},"min":{"1min":0.52600000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.4379999999999999,"5min":1.825,"15min":1.825},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.48499999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0469999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.82499999999999996}]},{"osd":8,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":4.0830000000000002,"5min":1.395,"15min":0.94699999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":55.694000000000003,"5min":55.694000000000003,"15min":55.694000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69199999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.88900000000000001,"5min":1.492,"15min":1.492},"last":0.65500000000000003}]},{"osd":10,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":2.149,"15min":2.222},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":17.527000000000001,"5min":27.920000000000002,"15min":27.920000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":8.1069999999999993,"15min":9.3320000000000007},"min":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.026,"5min":166.16200000000001,"15min":166.16200000000001},"last":0.80800000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.755},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.496,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84999999999999998,"5min":1.393,"15min":1.393},"last":0.60999999999999999}]}]}],"pool_statfs":[]}} 2024-09-15T01:34:26.822 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:34:27.099 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:34:27.099 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":138,"fsid":"d4803586-ed44-4154-8ee1-6b95ec3e17e7","created":"2024-09-15T01:32:07.285177+0000","modified":"2024-09-15T01:34:21.353291+0000","last_up_change":"2024-09-15T01:32:14.443323+0000","last_in_change":"2024-09-15T01:33:08.068993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:32:15.118634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:32:21.824131+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bec41583-a0a4-4712-a9ec-eec1e88fb60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":28861},{"type":"v1","addr":"172.21.15.42:6809","nonce":28861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":28861},{"type":"v1","addr":"172.21.15.42:6811","nonce":28861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":28861},{"type":"v1","addr":"172.21.15.42:6815","nonce":28861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":28861},{"type":"v1","addr":"172.21.15.42:6813","nonce":28861}]},"public_addr":"172.21.15.42:6809/28861","cluster_addr":"172.21.15.42:6811/28861","heartbeat_back_addr":"172.21.15.42:6815/28861","heartbeat_front_addr":"172.21.15.42:6813/28861","state":["exists","up"]},{"osd":1,"uuid":"2cca66be-8685-4a38-abf8-c864cd5434da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":28508},{"type":"v1","addr":"172.21.15.115:6809","nonce":28508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":28508},{"type":"v1","addr":"172.21.15.115:6811","nonce":28508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6817","nonce":28508},{"type":"v1","addr":"172.21.15.115:6819","nonce":28508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6813","nonce":28508},{"type":"v1","addr":"172.21.15.115:6815","nonce":28508}]},"public_addr":"172.21.15.115:6809/28508","cluster_addr":"172.21.15.115:6811/28508","heartbeat_back_addr":"172.21.15.115:6819/28508","heartbeat_front_addr":"172.21.15.115:6815/28508","state":["exists","up"]},{"osd":2,"uuid":"b8756aac-3a3e-41ba-a6ef-497063cb44df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":28732},{"type":"v1","addr":"172.21.15.152:6809","nonce":28732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":28732},{"type":"v1","addr":"172.21.15.152:6811","nonce":28732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":28732},{"type":"v1","addr":"172.21.15.152:6816","nonce":28732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":28732},{"type":"v1","addr":"172.21.15.152:6813","nonce":28732}]},"public_addr":"172.21.15.152:6809/28732","cluster_addr":"172.21.15.152:6811/28732","heartbeat_back_addr":"172.21.15.152:6816/28732","heartbeat_front_addr":"172.21.15.152:6813/28732","state":["exists","up"]},{"osd":3,"uuid":"554875f5-74c0-4b58-80ee-6ba93748f104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":28835},{"type":"v1","addr":"172.21.15.175:6803","nonce":28835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":28835},{"type":"v1","addr":"172.21.15.175:6805","nonce":28835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":28835},{"type":"v1","addr":"172.21.15.175:6809","nonce":28835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":28835},{"type":"v1","addr":"172.21.15.175:6807","nonce":28835}]},"public_addr":"172.21.15.175:6803/28835","cluster_addr":"172.21.15.175:6805/28835","heartbeat_back_addr":"172.21.15.175:6809/28835","heartbeat_front_addr":"172.21.15.175:6807/28835","state":["exists","up"]},{"osd":4,"uuid":"3fe2c05a-4c22-4dd7-a8c6-282155c82eb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":28860},{"type":"v1","addr":"172.21.15.42:6801","nonce":28860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":28860},{"type":"v1","addr":"172.21.15.42:6803","nonce":28860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":28860},{"type":"v1","addr":"172.21.15.42:6807","nonce":28860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":28860},{"type":"v1","addr":"172.21.15.42:6805","nonce":28860}]},"public_addr":"172.21.15.42:6801/28860","cluster_addr":"172.21.15.42:6803/28860","heartbeat_back_addr":"172.21.15.42:6807/28860","heartbeat_front_addr":"172.21.15.42:6805/28860","state":["exists","up"]},{"osd":5,"uuid":"a301d1ce-3e31-4782-ade3-f9c0f1e6a95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":28509},{"type":"v1","addr":"172.21.15.115:6801","nonce":28509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":28509},{"type":"v1","addr":"172.21.15.115:6803","nonce":28509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":28509},{"type":"v1","addr":"172.21.15.115:6807","nonce":28509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":28509},{"type":"v1","addr":"172.21.15.115:6805","nonce":28509}]},"public_addr":"172.21.15.115:6801/28509","cluster_addr":"172.21.15.115:6803/28509","heartbeat_back_addr":"172.21.15.115:6807/28509","heartbeat_front_addr":"172.21.15.115:6805/28509","state":["exists","up"]},{"osd":6,"uuid":"c7d180be-21c5-4674-a9f2-10a1c7794871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":28731},{"type":"v1","addr":"172.21.15.152:6801","nonce":28731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":28731},{"type":"v1","addr":"172.21.15.152:6803","nonce":28731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":28731},{"type":"v1","addr":"172.21.15.152:6807","nonce":28731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":28731},{"type":"v1","addr":"172.21.15.152:6805","nonce":28731}]},"public_addr":"172.21.15.152:6801/28731","cluster_addr":"172.21.15.152:6803/28731","heartbeat_back_addr":"172.21.15.152:6807/28731","heartbeat_front_addr":"172.21.15.152:6805/28731","state":["exists","up"]},{"osd":7,"uuid":"4d2429ea-772f-4fb9-a421-240f1399f16c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":28836},{"type":"v1","addr":"172.21.15.175:6811","nonce":28836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":28836},{"type":"v1","addr":"172.21.15.175:6813","nonce":28836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":28836},{"type":"v1","addr":"172.21.15.175:6817","nonce":28836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":28836},{"type":"v1","addr":"172.21.15.175:6815","nonce":28836}]},"public_addr":"172.21.15.175:6811/28836","cluster_addr":"172.21.15.175:6813/28836","heartbeat_back_addr":"172.21.15.175:6817/28836","heartbeat_front_addr":"172.21.15.175:6815/28836","state":["exists","up"]},{"osd":8,"uuid":"de7389ab-a29e-4177-9e94-484fc6a94102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":28862},{"type":"v1","addr":"172.21.15.42:6817","nonce":28862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":28862},{"type":"v1","addr":"172.21.15.42:6819","nonce":28862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":28862},{"type":"v1","addr":"172.21.15.42:6823","nonce":28862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":28862},{"type":"v1","addr":"172.21.15.42:6821","nonce":28862}]},"public_addr":"172.21.15.42:6817/28862","cluster_addr":"172.21.15.42:6819/28862","heartbeat_back_addr":"172.21.15.42:6823/28862","heartbeat_front_addr":"172.21.15.42:6821/28862","state":["exists","up"]},{"osd":9,"uuid":"4b4b44da-97d6-41f2-8fc6-57680b10aeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":28510},{"type":"v1","addr":"172.21.15.115:6814","nonce":28510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":28510},{"type":"v1","addr":"172.21.15.115:6818","nonce":28510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":28510},{"type":"v1","addr":"172.21.15.115:6823","nonce":28510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":28510},{"type":"v1","addr":"172.21.15.115:6821","nonce":28510}]},"public_addr":"172.21.15.115:6814/28510","cluster_addr":"172.21.15.115:6818/28510","heartbeat_back_addr":"172.21.15.115:6823/28510","heartbeat_front_addr":"172.21.15.115:6821/28510","state":["exists","up"]},{"osd":10,"uuid":"5f285a07-9057-4575-b911-b3977acd397c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":28733},{"type":"v1","addr":"172.21.15.152:6817","nonce":28733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":28733},{"type":"v1","addr":"172.21.15.152:6819","nonce":28733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":28733},{"type":"v1","addr":"172.21.15.152:6824","nonce":28733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":28733},{"type":"v1","addr":"172.21.15.152:6821","nonce":28733}]},"public_addr":"172.21.15.152:6817/28733","cluster_addr":"172.21.15.152:6819/28733","heartbeat_back_addr":"172.21.15.152:6824/28733","heartbeat_front_addr":"172.21.15.152:6821/28733","state":["exists","up"]},{"osd":11,"uuid":"dacc0f3f-a598-4055-92f4-d41102ceca83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":28837},{"type":"v1","addr":"172.21.15.175:6827","nonce":28837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":28837},{"type":"v1","addr":"172.21.15.175:6829","nonce":28837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":28837},{"type":"v1","addr":"172.21.15.175:6833","nonce":28837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":28837},{"type":"v1","addr":"172.21.15.175:6831","nonce":28837}]},"public_addr":"172.21.15.175:6827/28837","cluster_addr":"172.21.15.175:6829/28837","heartbeat_back_addr":"172.21.15.175:6833/28837","heartbeat_front_addr":"172.21.15.175:6831/28837","state":["exists","up"]},{"osd":12,"uuid":"d218197b-d0db-410c-abff-8b849a21a555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":28866},{"type":"v1","addr":"172.21.15.42:6825","nonce":28866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":28866},{"type":"v1","addr":"172.21.15.42:6827","nonce":28866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":28866},{"type":"v1","addr":"172.21.15.42:6831","nonce":28866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":28866},{"type":"v1","addr":"172.21.15.42:6829","nonce":28866}]},"public_addr":"172.21.15.42:6825/28866","cluster_addr":"172.21.15.42:6827/28866","heartbeat_back_addr":"172.21.15.42:6831/28866","heartbeat_front_addr":"172.21.15.42:6829/28866","state":["exists","up"]},{"osd":13,"uuid":"c9bf5faf-f2ee-482e-b903-28c7718e2063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":28511},{"type":"v1","addr":"172.21.15.115:6825","nonce":28511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":28511},{"type":"v1","addr":"172.21.15.115:6827","nonce":28511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":28511},{"type":"v1","addr":"172.21.15.115:6831","nonce":28511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":28511},{"type":"v1","addr":"172.21.15.115:6829","nonce":28511}]},"public_addr":"172.21.15.115:6825/28511","cluster_addr":"172.21.15.115:6827/28511","heartbeat_back_addr":"172.21.15.115:6831/28511","heartbeat_front_addr":"172.21.15.115:6829/28511","state":["exists","up"]},{"osd":14,"uuid":"cc3f19cf-5a97-440b-a953-7727e6ed6a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6823","nonce":28734},{"type":"v1","addr":"172.21.15.152:6825","nonce":28734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":28734},{"type":"v1","addr":"172.21.15.152:6827","nonce":28734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":28734},{"type":"v1","addr":"172.21.15.152:6831","nonce":28734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":28734},{"type":"v1","addr":"172.21.15.152:6829","nonce":28734}]},"public_addr":"172.21.15.152:6825/28734","cluster_addr":"172.21.15.152:6827/28734","heartbeat_back_addr":"172.21.15.152:6831/28734","heartbeat_front_addr":"172.21.15.152:6829/28734","state":["exists","up"]},{"osd":15,"uuid":"51290c23-e86d-4e05-9ac8-ca53d6a1eb80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":28838},{"type":"v1","addr":"172.21.15.175:6819","nonce":28838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":28838},{"type":"v1","addr":"172.21.15.175:6821","nonce":28838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":28838},{"type":"v1","addr":"172.21.15.175:6825","nonce":28838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":28838},{"type":"v1","addr":"172.21.15.175:6823","nonce":28838}]},"public_addr":"172.21.15.175:6819/28838","cluster_addr":"172.21.15.175:6821/28838","heartbeat_back_addr":"172.21.15.175:6825/28838","heartbeat_front_addr":"172.21.15.175:6823/28838","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:15.503649+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:25.772185+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:32.378075+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:25.443004+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:23.634325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.175620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.608100+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.434930+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:20.501276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:13.348208+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:41.326775+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.748721+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:30.563588+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:27.752979+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:34:20.712532+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:33:32.885128+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[10,8]}],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":25,"length":1},{"begin":31,"length":1},{"begin":41,"length":1},{"begin":44,"length":1},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":58,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:34:28.112 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T01:34:28.113 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:28.256 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:28.256 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:28.256 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:28.267 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T01:34:28.544 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T01:34:28.556 INFO:tasks.ceph:Scrubbing osd.4 2024-09-15T01:34:28.557 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:28.685 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:28.686 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:28.696 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-15T01:34:28.973 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 4 to deep-scrub 2024-09-15T01:34:28.986 INFO:tasks.ceph:Scrubbing osd.8 2024-09-15T01:34:28.986 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:29.115 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:29.115 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:29.115 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:29.125 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-09-15T01:34:29.405 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 8 to deep-scrub 2024-09-15T01:34:29.418 INFO:tasks.ceph:Scrubbing osd.12 2024-09-15T01:34:29.418 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:29.547 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:29.547 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:29.547 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:29.558 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2024-09-15T01:34:29.837 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 12 to deep-scrub 2024-09-15T01:34:29.849 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T01:34:29.850 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:29.980 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:29.981 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:29.991 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T01:34:30.273 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T01:34:30.286 INFO:tasks.ceph:Scrubbing osd.5 2024-09-15T01:34:30.286 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:30.418 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:30.419 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:30.429 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-15T01:34:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 5 to deep-scrub 2024-09-15T01:34:31.561 INFO:tasks.ceph:Scrubbing osd.9 2024-09-15T01:34:31.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:31.697 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:31.698 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:31.698 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:31.709 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-09-15T01:34:31.988 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 9 to deep-scrub 2024-09-15T01:34:32.001 INFO:tasks.ceph:Scrubbing osd.13 2024-09-15T01:34:32.002 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:32.131 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:32.132 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:32.142 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2024-09-15T01:34:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 13 to deep-scrub 2024-09-15T01:34:32.435 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T01:34:32.435 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:32.566 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:32.567 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:32.577 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T01:34:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T01:34:32.870 INFO:tasks.ceph:Scrubbing osd.6 2024-09-15T01:34:32.870 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:33.003 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:33.003 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:33.013 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-15T01:34:33.294 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 6 to deep-scrub 2024-09-15T01:34:33.307 INFO:tasks.ceph:Scrubbing osd.10 2024-09-15T01:34:33.307 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:33.438 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:33.438 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:33.449 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2024-09-15T01:34:33.730 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 10 to deep-scrub 2024-09-15T01:34:33.744 INFO:tasks.ceph:Scrubbing osd.14 2024-09-15T01:34:33.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:33.877 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:33.877 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:33.877 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:33.887 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2024-09-15T01:34:34.164 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 14 to deep-scrub 2024-09-15T01:34:34.177 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T01:34:34.178 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:34.307 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:34.307 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:34.307 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:34.318 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T01:34:34.596 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T01:34:34.609 INFO:tasks.ceph:Scrubbing osd.7 2024-09-15T01:34:34.609 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:34.739 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:34.739 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:34.739 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:34.749 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-15T01:34:35.030 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 7 to deep-scrub 2024-09-15T01:34:35.043 INFO:tasks.ceph:Scrubbing osd.11 2024-09-15T01:34:35.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:35.173 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:35.174 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:35.185 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2024-09-15T01:34:35.464 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 11 to deep-scrub 2024-09-15T01:34:35.477 INFO:tasks.ceph:Scrubbing osd.15 2024-09-15T01:34:35.477 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:34:35.607 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-15T01:34:35.608 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:34:35.608 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-15T01:34:35.618 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2024-09-15T01:34:35.901 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 15 to deep-scrub 2024-09-15T01:34:35.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:34:36.195 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:34:36.196 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:34:36.206 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-15T01:34:36.179760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":5,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":12771816,"kb_used_data":1486102280,"kb_used_omap":52164,"kb_used_meta":0,"kb_avail":1486102040,"statfs":{"total":1534846828544,"available":1521768488960,"internally_reserved":245760,"allocated":1521768734720,"data_stored":1521768734720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53415998,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.019943"},"pg_stats":[{"pgid":"1.0","version":"106'114","reported_seq":280,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T01:34:32.969056+0000","last_change":"2024-09-15T01:34:32.968990+0000","last_active":"2024-09-15T01:34:32.969056+0000","last_peered":"2024-09-15T01:34:32.969056+0000","last_clean":"2024-09-15T01:34:32.969056+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:34:32.969056+0000","last_undegraded":"2024-09-15T01:34:32.969056+0000","last_fullsized":"2024-09-15T01:34:32.969056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'114","last_scrub_stamp":"2024-09-15T01:34:32.968932+0000","last_deep_scrub":"106'114","last_deep_scrub_stamp":"2024-09-15T01:34:32.968932+0000","last_clean_scrub_stamp":"2024-09-15T01:34:32.968932+0000","objects_scrubbed":2,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:37.232860+0000","scrub_duration":0.0040350259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"105'2","reported_seq":143,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-15T01:34:33.170183+0000","last_change":"2024-09-15T01:34:32.454248+0000","last_active":"2024-09-15T01:34:33.170183+0000","last_peered":"2024-09-15T01:34:33.170183+0000","last_clean":"2024-09-15T01:34:33.170183+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:34:33.170183+0000","last_undegraded":"2024-09-15T01:34:33.170183+0000","last_fullsized":"2024-09-15T01:34:33.170183+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'2","last_scrub_stamp":"2024-09-15T01:34:32.454190+0000","last_deep_scrub":"105'2","last_deep_scrub_stamp":"2024-09-15T01:34:32.454190+0000","last_clean_scrub_stamp":"2024-09-15T01:34:32.454190+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:59.069162+0000","scrub_duration":0.002393807,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":84,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-15T01:34:34.315805+0000","last_change":"2024-09-15T01:34:34.315741+0000","last_active":"2024-09-15T01:34:34.315805+0000","last_peered":"2024-09-15T01:34:34.315805+0000","last_clean":"2024-09-15T01:34:34.315805+0000","last_became_active":"2024-09-15T01:33:23.083242+0000","last_became_peered":"2024-09-15T01:33:23.083242+0000","last_unstale":"2024-09-15T01:34:34.315805+0000","last_undegraded":"2024-09-15T01:34:34.315805+0000","last_fullsized":"2024-09-15T01:34:34.315805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:34:34.315649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:34:34.315649+0000","last_clean_scrub_stamp":"2024-09-15T01:34:34.315649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:50.156625+0000","scrub_duration":0.0018209389999999999,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8],"acting":[10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T01:34:32.421682+0000","last_change":"2024-09-15T01:33:50.941845+0000","last_active":"2024-09-15T01:34:32.421682+0000","last_peered":"2024-09-15T01:34:32.421682+0000","last_clean":"2024-09-15T01:34:32.421682+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:34:32.421682+0000","last_undegraded":"2024-09-15T01:34:32.421682+0000","last_fullsized":"2024-09-15T01:34:32.421682+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:50.941790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:50.941790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.0017165430000000001,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":137,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-15T01:34:33.680190+0000","last_change":"2024-09-15T01:33:46.870657+0000","last_active":"2024-09-15T01:34:33.680190+0000","last_peered":"2024-09-15T01:34:33.680190+0000","last_clean":"2024-09-15T01:34:33.680190+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:34:33.680190+0000","last_undegraded":"2024-09-15T01:34:33.680190+0000","last_fullsized":"2024-09-15T01:34:33.680190+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-15T01:33:46.870584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:46.870584+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.0027185820000000002,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":143,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T01:34:30.312906+0000","last_change":"2024-09-15T01:34:30.312873+0000","last_active":"2024-09-15T01:34:30.312906+0000","last_peered":"2024-09-15T01:34:30.312906+0000","last_clean":"2024-09-15T01:34:30.312906+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:34:30.312906+0000","last_undegraded":"2024-09-15T01:34:30.312906+0000","last_fullsized":"2024-09-15T01:34:30.312906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:34:30.312823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:34:30.312823+0000","last_clean_scrub_stamp":"2024-09-15T01:34:30.312823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:56.871066+0000","scrub_duration":0.0019107390000000001,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-15T01:34:33.625171+0000","last_change":"2024-09-15T01:33:41.543896+0000","last_active":"2024-09-15T01:34:33.625171+0000","last_peered":"2024-09-15T01:34:33.625171+0000","last_clean":"2024-09-15T01:34:33.625171+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:34:33.625171+0000","last_undegraded":"2024-09-15T01:34:33.625171+0000","last_fullsized":"2024-09-15T01:34:33.625171+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:41.543850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:41.543850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.0015955419999999999,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T01:34:32.484640+0000","last_change":"2024-09-15T01:34:32.484611+0000","last_active":"2024-09-15T01:34:32.484640+0000","last_peered":"2024-09-15T01:34:32.484640+0000","last_clean":"2024-09-15T01:34:32.484640+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:34:32.484640+0000","last_undegraded":"2024-09-15T01:34:32.484640+0000","last_fullsized":"2024-09-15T01:34:32.484640+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:34:32.484556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:34:32.484556+0000","last_clean_scrub_stamp":"2024-09-15T01:34:32.484556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:37.372594+0000","scrub_duration":0.0017780090000000001,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":131,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T01:34:30.598495+0000","last_change":"2024-09-15T01:33:38.175676+0000","last_active":"2024-09-15T01:34:30.598495+0000","last_peered":"2024-09-15T01:34:30.598495+0000","last_clean":"2024-09-15T01:34:30.598495+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:34:30.598495+0000","last_undegraded":"2024-09-15T01:34:30.598495+0000","last_fullsized":"2024-09-15T01:34:30.598495+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:33:38.175625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:22.121120+0000","last_clean_scrub_stamp":"2024-09-15T01:33:38.175625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:44.144155+0000","scrub_duration":0.001339526,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798060,"kb_used_data":92881568,"kb_used_omap":3956,"kb_used_meta":0,"kb_avail":92881556,"statfs":{"total":95927926784,"available":95110713344,"internally_reserved":12288,"allocated":95110725632,"data_stored":95110725632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4051565,"internal_metadata":0},"hb_peers":[1,2,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.56699999999999995}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":16.196999999999999,"5min":27.753,"15min":29.678999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":299.721,"5min":574.26900000000001,"15min":574.26900000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":18.404,"5min":4.0270000000000001,"15min":1.631},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":342.08300000000003,"5min":342.08300000000003,"15min":342.08300000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:33:20 2024","interfaces":[{"interface":"back","average":{"1min":32.810000000000002,"5min":32.810000000000002,"15min":32.810000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":644.97699999999998,"5min":644.97699999999998,"15min":644.97699999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":805.39499999999998}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.71399999999999997,"15min":0.73999999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.69299999999999995,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.372},{"interface":"front","average":{"1min":11.887,"5min":2.8730000000000002,"15min":1.371},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":214.994,"5min":214.994,"15min":214.994},"last":0.34499999999999997}]},{"osd":5,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.62},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.435}]},{"osd":6,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.499}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82899999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":10.272,"5min":2.625,"15min":1.351},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":173.11099999999999,"5min":173.11099999999999,"15min":173.11099999999999},"last":0.53500000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":6.056,"15min":6.9420000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.589,"5min":134.12200000000001,"15min":134.12200000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":45.521000000000001,"5min":9.7249999999999996,"15min":3.7589999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":846.97400000000005,"5min":846.97400000000005,"15min":846.97400000000005},"last":0.58699999999999997}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.56599999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.5649999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":17.137,"5min":4.0940000000000003,"15min":1.9199999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":313.495,"5min":313.495,"15min":313.495},"last":0.55100000000000005}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.70399999999999996}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81399999999999995,"15min":0.81299999999999994},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.60299999999999998}]}]},{"osd":9,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798068,"kb_used_data":92881564,"kb_used_omap":1551,"kb_used_meta":0,"kb_avail":92881548,"statfs":{"total":95927926784,"available":95110705152,"internally_reserved":16384,"allocated":95110721536,"data_stored":95110721536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588963,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.375},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.46999999999999997},"min":{"1min":0.38700000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.32600000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":23.486999999999998,"5min":5.2149999999999999,"15min":2.1699999999999999},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":462.93700000000001,"5min":462.93700000000001,"15min":462.93700000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.94399999999999995,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.57299999999999995}]},{"osd":7,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84399999999999997,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89800000000000002,"5min":1.97,"15min":1.97},"last":0.54700000000000004}]},{"osd":8,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.0720000000000001,"15min":1.1519999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":10.916,"15min":10.916},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.623,"5min":1.218,"15min":1.3169999999999999},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":13.147,"15min":13.147},"last":0.43099999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":5.8819999999999997,"5min":1.702,"15min":1.006},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":127.033,"5min":127.033,"15min":127.033},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.90600000000000003,"15min":0.95299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92900000000000005,"5min":6.4539999999999997,"15min":6.4539999999999997},"last":0.56299999999999994}]},{"osd":11,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":10.221,"5min":2.5840000000000001,"15min":1.3109999999999999},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":229.28899999999999,"5min":229.28899999999999,"15min":229.28899999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.88700000000000001,"15min":0.93100000000000005},"min":{"1min":0.42199999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.874,"5min":6.9489999999999998,"15min":6.9489999999999998},"last":0.495}]},{"osd":12,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":23.875,"15min":27.75},"min":{"1min":0.47899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97699999999999998,"5min":495.95800000000003,"15min":495.95800000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":43.860999999999997,"15min":51.058999999999997},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":914.18499999999995,"15min":914.18499999999995},"last":0.47599999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":2.7599999999999998,"5min":1.595,"15min":1.401},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":49.841000000000001,"5min":49.841000000000001,"15min":49.841000000000001},"last":0.51000000000000001}]},{"osd":14,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86099999999999999,"5min":1.96,"15min":1.96},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":174.88800000000001}]},{"osd":15,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98799999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91000000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.53000000000000003}]}]},{"osd":3,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798072,"kb_used_data":92881560,"kb_used_omap":2130,"kb_used_meta":0,"kb_avail":92881544,"statfs":{"total":95927926784,"available":95110701056,"internally_reserved":16384,"allocated":95110717440,"data_stored":95110717440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2181833,"internal_metadata":0},"hb_peers":[0,2,4,6,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.365,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.42699999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.91700000000000004}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.57699999999999996,"5min":0.312,"15min":0.312},"max":{"1min":0.99099999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.65700000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.81399999999999995,"15min":0.84699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1850000000000001,"5min":7.4720000000000004,"15min":7.4720000000000004},"last":0.439},{"interface":"front","average":{"1min":0.60199999999999998,"5min":7.5629999999999997,"15min":8.7230000000000008},"min":{"1min":0.41099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.1100000000000001,"5min":182.87,"15min":182.87},"last":0.34300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.5,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.49199999999999999},{"interface":"front","average":{"1min":28.616,"5min":6.2119999999999997,"15min":2.4780000000000002},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":501.65800000000002,"5min":501.65800000000002,"15min":501.65800000000002},"last":0.64600000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":24.288,"5min":24.288,"15min":24.288},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":500.678,"5min":500.678,"15min":500.678},"last":0.55100000000000005},{"interface":"front","average":{"1min":35.298999999999999,"5min":35.298999999999999,"15min":35.298999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":500.79899999999998,"5min":500.79899999999998,"15min":500.79899999999998},"last":0.52200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":11.316000000000001,"5min":11.316000000000001,"15min":11.316000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":200.40799999999999,"5min":200.40799999999999,"15min":200.40799999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.48399999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.378},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.40999999999999998}]},{"osd":12,"last update":"Sun Sep 15 01:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.39000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.3300000000000001,"5min":1.762,"15min":1.762},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.177,"5min":1.417,"15min":1.417},"last":0.55000000000000004}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.78400000000000003,"15min":0.76800000000000002},"min":{"1min":0.67900000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.427,"5min":1.637,"15min":1.637},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0169999999999999,"15min":1.0389999999999999},"min":{"1min":0.63200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.476,"5min":7.4500000000000002,"15min":7.4500000000000002},"last":0.58399999999999996}]},{"osd":14,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.502},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.51500000000000001}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.63300000000000001}]}]},{"osd":0,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798192,"kb_used_data":92881436,"kb_used_omap":5387,"kb_used_meta":0,"kb_avail":92881424,"statfs":{"total":95927926784,"available":95110578176,"internally_reserved":12288,"allocated":95110590464,"data_stored":95110590464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5517280,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.497},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.113,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":31.643999999999998,"15min":36.808999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.488,"5min":814.68299999999999,"15min":814.68299999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.63700000000000001,"15min":0.65200000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1890000000000001,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.43099999999999999},{"interface":"front","average":{"1min":40.415999999999997,"5min":8.5380000000000003,"15min":3.2250000000000001},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":799.04999999999995,"5min":799.04999999999995,"15min":799.04999999999995},"last":0.36499999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69399999999999995,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.2769999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.264,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.58499999999999996}]},{"osd":7,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":46.667999999999999,"5min":9.8330000000000002,"15min":3.694},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":920.58399999999995,"5min":920.58399999999995,"15min":920.58399999999995},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.41599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0629999999999999,"5min":1.304,"15min":1.304},"last":0.53700000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.222,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.51100000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":414.40899999999999}]},{"osd":12,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.52300000000000002}]},{"osd":13,"last update":"Sun Sep 15 01:33:35 2024","interfaces":[{"interface":"back","average":{"1min":25.651,"5min":25.651,"15min":25.651},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":501.07400000000001,"5min":501.07400000000001,"15min":501.07400000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":44.5,"5min":44.5,"15min":44.5},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":501.08800000000002,"5min":501.08800000000002,"15min":501.08800000000002},"last":0.66100000000000003}]},{"osd":15,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.403,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.64700000000000002}]}]},{"osd":13,"up_from":21,"seq":90194313248,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799284,"kb_used_data":92880352,"kb_used_omap":4487,"kb_used_meta":0,"kb_avail":92880332,"statfs":{"total":95927926784,"available":95109459968,"internally_reserved":20480,"allocated":95109480448,"data_stored":95109480448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4595382,"internal_metadata":0},"hb_peers":[2,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":1.0049999999999999,"15min":1.0660000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.109,"5min":8.1129999999999995,"15min":8.1129999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.997,"15min":1.0600000000000001},"min":{"1min":0.38700000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1739999999999999,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":0.69999999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":19.710000000000001,"5min":19.710000000000001,"15min":19.710000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":342.56099999999998,"5min":342.56099999999998,"15min":342.56099999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":28.227,"5min":28.227,"15min":28.227},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":495.40800000000002,"5min":495.40800000000002,"15min":495.40800000000002},"last":0.60099999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":24.440999999999999,"5min":5.6139999999999999,"15min":2.476},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":572.322,"5min":572.322,"15min":572.322},"last":0.442},{"interface":"front","average":{"1min":41.835999999999999,"5min":9.0950000000000006,"15min":3.6379999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":990.04700000000003,"5min":990.04700000000003,"15min":990.04700000000003},"last":0.47399999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46999999999999997,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.58899999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.33600000000000002}]},{"osd":6,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":18.945,"5min":18.945,"15min":18.945},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":293.34800000000001,"5min":293.34800000000001,"15min":293.34800000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.45700000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.91400000000000003,"15min":0.95299999999999996},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.488,"5min":6.4779999999999998,"15min":6.4779999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":21.263000000000002,"5min":5.024,"15min":2.3180000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":493.435,"5min":493.435,"15min":493.435},"last":0.53900000000000003}]},{"osd":8,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.85499999999999998,"15min":0.89300000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90000000000000002,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":2.6000000000000001,"15min":2.9249999999999998},"min":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94999999999999996,"5min":44.985999999999997,"15min":44.985999999999997},"last":0.42299999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.51600000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.50800000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.499}]},{"osd":14,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73099999999999998,"15min":0.72399999999999998},"min":{"1min":0.56000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.80400000000000005,"15min":0.80800000000000005},"min":{"1min":0.56699999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2669999999999999,"5min":1.474,"15min":1.474},"last":0.74199999999999999}]},{"osd":15,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":46.790999999999997,"5min":9.9949999999999992,"15min":3.8620000000000001},"min":{"1min":0.53400000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":806.28599999999994,"5min":806.28599999999994,"15min":806.28599999999994},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.52500000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2989999999999999,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.63700000000000001}]}]},{"osd":2,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798772,"kb_used_data":92880860,"kb_used_omap":4396,"kb_used_meta":0,"kb_avail":92880844,"statfs":{"total":95927926784,"available":95109984256,"internally_reserved":16384,"allocated":95110000640,"data_stored":95110000640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4501799,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95199999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.64200000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86599999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.38400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.503}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":46.487000000000002,"5min":9.6280000000000001,"15min":3.4849999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":829.03399999999999,"5min":829.03399999999999,"15min":829.03399999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.43099999999999999,"15min":0.41099999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":857.59500000000003}]},{"osd":7,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":39.168999999999997,"5min":13.167999999999999,"15min":8.8339999999999996},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":655.90899999999999,"5min":655.90899999999999,"15min":655.90899999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59599999999999997}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0329999999999999,"5min":1.329,"15min":1.329},"last":0.48499999999999999},{"interface":"front","average":{"1min":54.841000000000001,"5min":11.481,"15min":4.2539999999999996},"min":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":976.00800000000004,"5min":976.00800000000004,"15min":976.00800000000004},"last":0.42999999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":340.62599999999998}]},{"osd":11,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.68700000000000006,"15min":0.63600000000000001},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":6.4240000000000004,"5min":6.4240000000000004,"15min":6.4240000000000004},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.74399999999999999,"15min":0.68999999999999995},"min":{"1min":0.47299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":6.4390000000000001,"5min":6.4390000000000001,"15min":6.4390000000000001},"last":0.56499999999999995}]},{"osd":12,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":47.031999999999996,"5min":9.9469999999999992,"15min":3.766},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":834.505,"5min":834.505,"15min":834.505},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97199999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.65600000000000003}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99199999999999999,"5min":1.363,"15min":1.363},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.68799999999999994}]},{"osd":14,"last update":"Sun Sep 15 01:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.67300000000000004}]}]},{"osd":15,"up_from":21,"seq":90194313248,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798192,"kb_used_data":92881440,"kb_used_omap":3197,"kb_used_meta":0,"kb_avail":92881424,"statfs":{"total":95927926784,"available":95110578176,"internally_reserved":16384,"allocated":95110594560,"data_stored":95110594560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3273974,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.79300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.498}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":12.098000000000001,"5min":2.8300000000000001,"15min":1.2849999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":162.16800000000001,"5min":162.16800000000001,"15min":162.16800000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":47.701000000000001,"5min":9.8719999999999999,"15min":3.5670000000000002},"min":{"1min":0.26600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":662.84000000000003,"5min":662.84000000000003,"15min":662.84000000000003},"last":0.53800000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64200000000000002,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57099999999999995,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.63900000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94899999999999995,"5min":1.385,"15min":1.385},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.69599999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":4.798,"15min":5.5069999999999997},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69999999999999996,"5min":90.775999999999996,"15min":90.775999999999996},"last":0.747},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.84899999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:33:36 2024","interfaces":[{"interface":"back","average":{"1min":50.158999999999999,"5min":50.158999999999999,"15min":50.158999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":894.27700000000004,"5min":894.27700000000004,"15min":894.27700000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":49.308,"5min":49.308,"15min":49.308},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":878.12199999999996,"5min":878.12199999999996,"15min":878.12199999999996},"last":0.65900000000000003}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":50.171999999999997,"5min":10.541,"15min":3.9359999999999999},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":645.34900000000005,"5min":645.34900000000005,"15min":645.34900000000005},"last":0.62},{"interface":"front","average":{"1min":0.69699999999999995,"5min":27.992000000000001,"15min":32.540999999999997},"min":{"1min":0.45900000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99099999999999999,"5min":542.06600000000003,"15min":542.06600000000003},"last":0.76200000000000001}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":1.603,"15min":1.603},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96199999999999997,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.71499999999999997}]},{"osd":14,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":1.163,"15min":1.163},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.75,"5min":12.249000000000001,"15min":14.164999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.224,"5min":246.56100000000001,"15min":246.56100000000001},"last":0.86199999999999999}]}]},{"osd":4,"up_from":21,"seq":90194313248,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798120,"kb_used_data":92881496,"kb_used_omap":3063,"kb_used_meta":0,"kb_avail":92881496,"statfs":{"total":95927926784,"available":95110651904,"internally_reserved":0,"allocated":95110651904,"data_stored":95110651904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3137008,"internal_metadata":0},"hb_peers":[0,3,5,6,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.441,"15min":0.437},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.43099999999999999,"5min":14.403,"15min":16.731999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60399999999999998,"5min":314.661,"15min":314.661},"last":0.27300000000000002}]},{"osd":1,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51300000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.747},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":1.127}]},{"osd":3,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.55900000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0940000000000001,"5min":1.179,"15min":1.179},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.45000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.45800000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":42.378,"5min":24.170000000000002,"15min":21.135000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":918.75699999999995,"5min":918.75699999999995,"15min":918.75699999999995},"last":0.69099999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":7.5110000000000001,"5min":2.1309999999999998,"15min":1.2350000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":142.26599999999999,"5min":142.26599999999999,"15min":142.26599999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.57899999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1719999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.66900000000000004}]},{"osd":7,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0840000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.871}]},{"osd":9,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78300000000000003,"15min":0.77900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.63400000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":40.198999999999998,"5min":8.7089999999999996,"15min":3.4609999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":828.54499999999996,"5min":828.54499999999996,"15min":828.54499999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":14.634,"5min":3.5920000000000001,"15min":1.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":291.08800000000002,"5min":291.08800000000002,"15min":291.08800000000002},"last":0.81499999999999995}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.443},{"interface":"front","average":{"1min":28.721,"5min":28.721,"15min":28.721},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":558.56700000000001,"5min":558.56700000000001,"15min":558.56700000000001},"last":0.48799999999999999}]},{"osd":12,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1799999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.39000000000000001}]},{"osd":13,"last update":"Sun Sep 15 01:34:33 2024","interfaces":[{"interface":"back","average":{"1min":21.696999999999999,"5min":4.9790000000000001,"15min":2.1930000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":441.02800000000002,"5min":441.02800000000002,"15min":441.02800000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81399999999999995,"15min":0.82099999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.77100000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.61099999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3169999999999999,"5min":1.325,"15min":1.325},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.90500000000000003,"15min":0.90700000000000003},"min":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.256,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.55300000000000005}]}]},{"osd":5,"up_from":21,"seq":90194313247,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798196,"kb_used_data":92881440,"kb_used_omap":3058,"kb_used_meta":0,"kb_avail":92881420,"statfs":{"total":95927926784,"available":95110574080,"internally_reserved":20480,"allocated":95110594560,"data_stored":95110594560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3132105,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.377}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.35499999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60399999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47799999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.628,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.48,"5min":1.524,"15min":1.524},"last":0.61599999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.70199999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":879.18399999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.57299999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92200000000000004,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.5}]},{"osd":13,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":36.902999999999999,"5min":7.8339999999999996,"15min":2.9889999999999999},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":764.33399999999995,"5min":764.33399999999995,"15min":764.33399999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.53200000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1899999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":17.335999999999999,"5min":22.786000000000001,"15min":23.693999999999999},"min":{"1min":0.496,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":346.49200000000002,"5min":492.697,"15min":492.697},"last":0.626}]},{"osd":15,"last update":"Sun Sep 15 01:34:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":11.316000000000001,"15min":13.064},"min":{"1min":0.54700000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6399999999999999,"5min":291.12599999999998,"15min":291.12599999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.77700000000000002,"15min":0.76100000000000001},"min":{"1min":0.61199999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.67000000000000004}]}]},{"osd":6,"up_from":21,"seq":90194313247,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798200,"kb_used_data":92881436,"kb_used_omap":3139,"kb_used_meta":0,"kb_avail":92881416,"statfs":{"total":95927926784,"available":95110569984,"internally_reserved":20480,"allocated":95110590464,"data_stored":95110590464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3214408,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.85699999999999998,"15min":0.88100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.165,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77600000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47899999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.878,"5min":1.264,"15min":1.264},"last":0.58999999999999997}]},{"osd":4,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.153,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.873}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77800000000000002,"5min":1.163,"15min":1.163},"last":0.56100000000000005}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.379,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83799999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.746,"15min":0.746},"last":0.73399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.86199999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":61.164000000000001,"5min":12.723000000000001,"15min":4.649},"min":{"1min":0.48099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":656.21000000000004,"5min":656.21000000000004,"15min":656.21000000000004},"last":0.77900000000000003}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":11.742000000000001,"5min":2.8580000000000001,"15min":1.3779999999999999},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":200.52799999999999,"5min":200.52799999999999,"15min":200.52799999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.76300000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82999999999999996,"5min":1.353,"15min":1.353},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.99199999999999999,"15min":1.036},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97999999999999998,"5min":5.8470000000000004,"15min":5.8470000000000004},"last":0.84599999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.55800000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.55200000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.034,"5min":1.212,"15min":1.212},"last":0.88200000000000001}]}]},{"osd":7,"up_from":21,"seq":90194313248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798064,"kb_used_data":92881564,"kb_used_omap":2579,"kb_used_meta":0,"kb_avail":92881552,"statfs":{"total":95927926784,"available":95110709248,"internally_reserved":12288,"allocated":95110721536,"data_stored":95110721536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2641293,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.48199999999999998,"15min":0.47099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":47.609999999999999,"15min":55.453000000000003},"min":{"1min":0.36699999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.69999999999999996,"5min":874.39499999999998,"15min":874.39499999999998},"last":0.47999999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":4.7350000000000003,"5min":4.7350000000000003,"15min":4.7350000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.34599999999999997}]},{"osd":4,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.58999999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":1.165,"15min":1.165},"last":0.624},{"interface":"front","average":{"1min":0.61399999999999999,"5min":1.0880000000000001,"15min":1.167},"min":{"1min":0.42799999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":5.6539999999999999,"15min":5.6539999999999999},"last":0.68700000000000006}]},{"osd":8,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.497}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":4.4359999999999999,"15min":5.069},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84899999999999998,"5min":62.991999999999997,"15min":62.991999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":389.18799999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67600000000000005,"15min":0.68400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":1.21,"15min":1.21},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":43.418999999999997,"15min":50.549999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93400000000000005,"5min":792.73599999999999,"15min":792.73599999999999},"last":0.63500000000000001}]},{"osd":13,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.091,"15min":1.157},"min":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92100000000000004,"5min":8.1489999999999991,"15min":8.1489999999999991},"last":0.73799999999999999}]},{"osd":14,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.47199999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.878,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":41.905000000000001,"15min":48.776000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.83099999999999996,"5min":773.18799999999999,"15min":773.18799999999999},"last":0.64800000000000002}]},{"osd":15,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.1220000000000001,"15min":1.216},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79000000000000004,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83399999999999996,"5min":1.018,"15min":1.018},"last":0.57699999999999996}]}]},{"osd":8,"up_from":21,"seq":90194313247,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798204,"kb_used_data":92881436,"kb_used_omap":3904,"kb_used_meta":0,"kb_avail":92881412,"statfs":{"total":95927926784,"available":95110565888,"internally_reserved":24576,"allocated":95110590464,"data_stored":95110590464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3998332,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.373},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.626,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.36299999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":24.794,"5min":24.794,"15min":24.794},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":413.66800000000001,"5min":413.66800000000001,"15min":413.66800000000001},"last":0.53800000000000003}]},{"osd":2,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.38500000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72499999999999998,"5min":1.093,"15min":1.093},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.44700000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.95599999999999996,"5min":1.073,"15min":1.073},"last":0.63700000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":30.347000000000001,"5min":6.5869999999999997,"15min":2.6269999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":505.29300000000001,"5min":505.29300000000001,"15min":505.29300000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":12.156000000000001,"5min":2.9510000000000001,"15min":1.417},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":196.00299999999999,"5min":196.00299999999999,"15min":196.00299999999999},"last":0.61599999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.46899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97699999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.75600000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.497}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0249999999999999,"5min":1.208,"15min":1.208},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.004,"5min":1.351,"15min":1.351},"last":0.67100000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.503,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.77400000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":1.054,"15min":1.054},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98999999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.628}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798072,"kb_used_data":92881564,"kb_used_omap":3283,"kb_used_meta":0,"kb_avail":92881544,"statfs":{"total":95927926784,"available":95110701056,"internally_reserved":20480,"allocated":95110721536,"data_stored":95110721536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3361981,"internal_metadata":0},"hb_peers":[3,5,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.899},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40500000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.435}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75700000000000001,"15min":0.752},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":3.9750000000000001,"5min":3.9750000000000001,"15min":3.9750000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":3.9039999999999999,"5min":4.3150000000000004,"15min":4.383},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":56.188000000000002,"5min":64.938999999999993,"15min":64.938999999999993},"last":0.57099999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.32600000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":4.7960000000000003,"5min":1.5009999999999999,"15min":0.95199999999999996},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":75.778999999999996,"5min":75.778999999999996,"15min":75.778999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.75900000000000001,"15min":0.78100000000000003},"min":{"1min":0.45300000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84599999999999997,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.55400000000000005}]},{"osd":8,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":23.452999999999999,"5min":23.452999999999999,"15min":23.452999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":434.37700000000001,"5min":434.37700000000001,"15min":434.37700000000001},"last":0.58799999999999997}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":44.508000000000003,"15min":51.82},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96299999999999997,"5min":500.92099999999999,"15min":500.92099999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.41399999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66800000000000004}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.627}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84199999999999997,"15min":0.87},"min":{"1min":0.503,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0229999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.65300000000000002}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.4460000000000002},{"interface":"front","average":{"1min":36.435000000000002,"5min":36.435000000000002,"15min":36.435000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":641.649,"5min":641.649,"15min":641.649},"last":2.4740000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.048},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79600000000000004}]},{"osd":15,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.93899999999999995,"15min":0.97299999999999998},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.036,"5min":6.0670000000000002,"15min":6.0670000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0089999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.83999999999999997}]}]},{"osd":11,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798064,"kb_used_data":92881564,"kb_used_omap":3336,"kb_used_meta":0,"kb_avail":92881552,"statfs":{"total":95927926784,"available":95110709248,"internally_reserved":12288,"allocated":95110721536,"data_stored":95110721536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3416642,"internal_metadata":0},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":46.469000000000001,"5min":46.469000000000001,"15min":46.469000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":873.95500000000004,"5min":873.95500000000004,"15min":873.95500000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.954,"5min":1.954,"15min":1.954},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":28.431999999999999,"5min":28.431999999999999,"15min":28.431999999999999},"last":0.50900000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.308}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":13.093,"5min":3.125,"15min":1.4630000000000001},"min":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":250.767,"5min":250.767,"15min":250.767},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84899999999999998,"5min":0.871,"15min":0.871},"last":0.39000000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.505},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.42199999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.51700000000000002}]},{"osd":6,"last update":"Sun Sep 15 01:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59399999999999997}]},{"osd":8,"last update":"Sun Sep 15 01:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79800000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":6.2300000000000004,"5min":1.671,"15min":0.91100000000000003},"min":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":101.952,"5min":101.952,"15min":101.952},"last":0.35199999999999998}]},{"osd":9,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.46899999999999997}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86899999999999999,"5min":1.042,"15min":1.042},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90200000000000002,"5min":2.262,"15min":2.262},"last":0.63800000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.79000000000000004,"15min":0.81799999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93400000000000005,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.82099999999999995,"15min":0.84699999999999998},"min":{"1min":0.45300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85999999999999999,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":0.42299999999999999}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":1.903,"5min":0.95599999999999996,"15min":0.79800000000000004},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":25.056999999999999,"5min":25.056999999999999,"15min":25.056999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.442,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.55400000000000005}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":4.4009999999999998,"5min":13.781000000000001,"15min":15.343999999999999},"min":{"1min":0.51200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":74.078000000000003,"5min":310.78199999999998,"15min":310.78199999999998},"last":0.625},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.436,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97099999999999997,"5min":2.238,"15min":2.238},"last":0.57299999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":12,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798192,"kb_used_data":92881436,"kb_used_omap":1472,"kb_used_meta":0,"kb_avail":92881424,"statfs":{"total":95927926784,"available":95110578176,"internally_reserved":12288,"allocated":95110590464,"data_stored":95110590464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1507919,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":20.391999999999999,"5min":4.4370000000000003,"15min":1.778},"min":{"1min":0.27800000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":480.024,"5min":480.024,"15min":480.024},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.435,"15min":0.44500000000000001},"min":{"1min":0.26400000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.68000000000000005,"5min":0.875,"15min":0.875},"last":0.32800000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":15.561999999999999,"5min":3.8639999999999999,"15min":1.9139999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":360.94400000000002,"5min":360.94400000000002,"15min":360.94400000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.751,"15min":0.79100000000000004},"min":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":3.706,"15min":3.706},"last":0.34999999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.753,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.42999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85099999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.52500000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.65800000000000003,"15min":0.67400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80200000000000005,"5min":1.409,"15min":1.409},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60199999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84199999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":20.039000000000001,"15min":23.303999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":488.69,"15min":488.69},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.70899999999999996,"15min":0.754},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63300000000000001,"5min":3.5419999999999998,"15min":3.5419999999999998},"last":0.46899999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.82899999999999996,"15min":0.85499999999999998},"min":{"1min":0.41499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.97599999999999998,"5min":3.6309999999999998,"15min":3.6309999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.85099999999999998,"5min":1.109,"15min":1.109},"last":0.67000000000000004}]},{"osd":11,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.64200000000000002,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86299999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":32.154000000000003,"5min":7.1440000000000001,"15min":2.976},"min":{"1min":0.46899999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":724.78499999999997,"5min":724.78499999999997,"15min":724.78499999999997},"last":0.53800000000000003}]},{"osd":13,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94399999999999995,"5min":1.145,"15min":1.145},"last":0.55200000000000005},{"interface":"front","average":{"1min":5.3300000000000001,"5min":1.5660000000000001,"15min":0.93799999999999994},"min":{"1min":0.40400000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":107.797,"5min":107.797,"15min":107.797},"last":0.499}]},{"osd":14,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.008,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.018,"5min":1.133,"15min":1.133},"last":0.63500000000000001}]},{"osd":15,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.78300000000000003,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.0409999999999999,"5min":1.712,"15min":1.712},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.80600000000000005,"15min":0.81799999999999995},"min":{"1min":0.52900000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.995,"5min":1.595,"15min":1.595},"last":0.629}]}]},{"osd":1,"up_from":21,"seq":90194313248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798064,"kb_used_data":92881564,"kb_used_omap":3218,"kb_used_meta":0,"kb_avail":92881552,"statfs":{"total":95927926784,"available":95110709248,"internally_reserved":12288,"allocated":95110721536,"data_stored":95110721536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3295514,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":55.167999999999999,"5min":11.429,"15min":4.1390000000000002},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.31},"max":{"1min":875.06100000000004,"5min":875.06100000000004,"15min":875.06100000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.754,"15min":0.754},"last":0.38900000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79100000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61299999999999999,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91100000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.65400000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.66400000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.29999999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79200000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":28.789000000000001,"15min":33.491},"min":{"1min":0.38400000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82999999999999996,"5min":635.46400000000006,"15min":635.46400000000006},"last":0.44800000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.53800000000000003},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71199999999999997,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.37},{"interface":"front","average":{"1min":0.495,"5min":0.80500000000000005,"15min":0.85699999999999998},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72699999999999998,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.42799999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":54.337000000000003,"5min":11.387,"15min":4.2290000000000001},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":806.31200000000001,"5min":806.31200000000001,"15min":806.31200000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68000000000000005,"15min":0.68600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83799999999999997,"5min":1.341,"15min":1.341},"last":0.59099999999999997}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.75,"15min":0.751},"min":{"1min":0.52600000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.4379999999999999,"5min":1.825,"15min":1.825},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.48499999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0469999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.57299999999999995}]},{"osd":8,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":4.0830000000000002,"5min":1.395,"15min":0.94699999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":55.694000000000003,"5min":55.694000000000003,"15min":55.694000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69199999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.88900000000000001,"5min":1.492,"15min":1.492},"last":0.47199999999999998}]},{"osd":10,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":2.149,"15min":2.222},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":17.527000000000001,"5min":27.920000000000002,"15min":27.920000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":8.1069999999999993,"15min":9.3320000000000007},"min":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.026,"5min":166.16200000000001,"15min":166.16200000000001},"last":0.627}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.496,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84999999999999998,"5min":1.393,"15min":1.393},"last":0.61099999999999999}]}]}],"pool_statfs":[]}} 2024-09-15T01:34:36.213 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-15T01:33:50.941790+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=33, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=34, tm_sec=27, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:34:36.213 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T01:33:46.870584+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=33, tm_sec=46, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=34, tm_sec=27, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:34:36.213 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T01:33:41.543850+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=33, tm_sec=41, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=34, tm_sec=27, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:34:36.213 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T01:33:38.175625+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=33, tm_sec=38, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=34, tm_sec=27, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:34:36.213 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T01:34:56.214 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:34:56.512 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T01:34:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T01:34:56.524 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-15T01:34:55.101691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":5,"num_per_pool_omap_osds":0,"kb":1498873856,"kb_used":12773152,"kb_used_data":1486100896,"kb_used_omap":52233,"kb_used_meta":0,"kb_avail":1486100704,"statfs":{"total":1534846828544,"available":1521767120896,"internally_reserved":196608,"allocated":1521767317504,"data_stored":1521767317504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":53486876,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001204"},"pg_stats":[{"pgid":"1.0","version":"106'114","reported_seq":285,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T01:34:43.693285+0000","last_change":"2024-09-15T01:34:32.968990+0000","last_active":"2024-09-15T01:34:43.693285+0000","last_peered":"2024-09-15T01:34:43.693285+0000","last_clean":"2024-09-15T01:34:43.693285+0000","last_became_active":"2024-09-15T01:32:16.749680+0000","last_became_peered":"2024-09-15T01:32:16.749680+0000","last_unstale":"2024-09-15T01:34:43.693285+0000","last_undegraded":"2024-09-15T01:34:43.693285+0000","last_fullsized":"2024-09-15T01:34:43.693285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'114","last_scrub_stamp":"2024-09-15T01:34:32.968932+0000","last_deep_scrub":"106'114","last_deep_scrub_stamp":"2024-09-15T01:34:32.968932+0000","last_clean_scrub_stamp":"2024-09-15T01:34:32.968932+0000","objects_scrubbed":2,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:37.232860+0000","scrub_duration":0.0040350259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"105'2","reported_seq":147,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T01:34:43.891710+0000","last_change":"2024-09-15T01:34:32.454248+0000","last_active":"2024-09-15T01:34:43.891710+0000","last_peered":"2024-09-15T01:34:43.891710+0000","last_clean":"2024-09-15T01:34:43.891710+0000","last_became_active":"2024-09-15T01:32:23.138744+0000","last_became_peered":"2024-09-15T01:32:23.138744+0000","last_unstale":"2024-09-15T01:34:43.891710+0000","last_undegraded":"2024-09-15T01:34:43.891710+0000","last_fullsized":"2024-09-15T01:34:43.891710+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'2","last_scrub_stamp":"2024-09-15T01:34:32.454190+0000","last_deep_scrub":"105'2","last_deep_scrub_stamp":"2024-09-15T01:34:32.454190+0000","last_clean_scrub_stamp":"2024-09-15T01:34:32.454190+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:59.069162+0000","scrub_duration":0.002393807,"objects_trimmed":0,"snaptrim_duration":0.00026468599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":88,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T01:34:43.693168+0000","last_change":"2024-09-15T01:34:34.315741+0000","last_active":"2024-09-15T01:34:43.693168+0000","last_peered":"2024-09-15T01:34:43.693168+0000","last_clean":"2024-09-15T01:34:43.693168+0000","last_became_active":"2024-09-15T01:33:23.083242+0000","last_became_peered":"2024-09-15T01:33:23.083242+0000","last_unstale":"2024-09-15T01:34:43.693168+0000","last_undegraded":"2024-09-15T01:34:43.693168+0000","last_fullsized":"2024-09-15T01:34:43.693168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:34:34.315649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:34:34.315649+0000","last_clean_scrub_stamp":"2024-09-15T01:34:34.315649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:50.156625+0000","scrub_duration":0.0018209389999999999,"objects_trimmed":0,"snaptrim_duration":0.000294834,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8],"acting":[10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":149,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T01:34:43.693257+0000","last_change":"2024-09-15T01:34:36.966329+0000","last_active":"2024-09-15T01:34:43.693257+0000","last_peered":"2024-09-15T01:34:43.693257+0000","last_clean":"2024-09-15T01:34:43.693257+0000","last_became_active":"2024-09-15T01:32:23.134656+0000","last_became_peered":"2024-09-15T01:32:23.134656+0000","last_unstale":"2024-09-15T01:34:43.693257+0000","last_undegraded":"2024-09-15T01:34:43.693257+0000","last_fullsized":"2024-09-15T01:34:43.693257+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:34:36.966271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:34:36.966271+0000","last_clean_scrub_stamp":"2024-09-15T01:34:36.966271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:03.347031+0000","scrub_duration":0.001479637,"objects_trimmed":0,"snaptrim_duration":0.00032431199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"142'3","reported_seq":153,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T01:34:43.203761+0000","last_change":"2024-09-15T01:34:37.788661+0000","last_active":"2024-09-15T01:34:43.203761+0000","last_peered":"2024-09-15T01:34:43.203761+0000","last_clean":"2024-09-15T01:34:43.203761+0000","last_became_active":"2024-09-15T01:32:23.155796+0000","last_became_peered":"2024-09-15T01:32:23.155796+0000","last_unstale":"2024-09-15T01:34:43.203761+0000","last_undegraded":"2024-09-15T01:34:43.203761+0000","last_fullsized":"2024-09-15T01:34:43.203761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'3","last_scrub_stamp":"2024-09-15T01:34:37.788604+0000","last_deep_scrub":"142'3","last_deep_scrub_stamp":"2024-09-15T01:34:37.788604+0000","last_clean_scrub_stamp":"2024-09-15T01:34:37.788604+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:59.201171+0000","scrub_duration":0.0032974549999999999,"objects_trimmed":0,"snaptrim_duration":0.00072483699999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T01:34:43.692971+0000","last_change":"2024-09-15T01:34:30.312873+0000","last_active":"2024-09-15T01:34:43.692971+0000","last_peered":"2024-09-15T01:34:43.692971+0000","last_clean":"2024-09-15T01:34:43.692971+0000","last_became_active":"2024-09-15T01:32:23.155045+0000","last_became_peered":"2024-09-15T01:32:23.155045+0000","last_unstale":"2024-09-15T01:34:43.692971+0000","last_undegraded":"2024-09-15T01:34:43.692971+0000","last_fullsized":"2024-09-15T01:34:43.692971+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:34:30.312823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:34:30.312823+0000","last_clean_scrub_stamp":"2024-09-15T01:34:30.312823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:56.871066+0000","scrub_duration":0.0019107390000000001,"objects_trimmed":0,"snaptrim_duration":0.00033500199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":149,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T01:34:43.203254+0000","last_change":"2024-09-15T01:34:37.705595+0000","last_active":"2024-09-15T01:34:43.203254+0000","last_peered":"2024-09-15T01:34:43.203254+0000","last_clean":"2024-09-15T01:34:43.203254+0000","last_became_active":"2024-09-15T01:32:23.134443+0000","last_became_peered":"2024-09-15T01:32:23.134443+0000","last_unstale":"2024-09-15T01:34:43.203254+0000","last_undegraded":"2024-09-15T01:34:43.203254+0000","last_fullsized":"2024-09-15T01:34:43.203254+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:34:37.705533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:34:37.705533+0000","last_clean_scrub_stamp":"2024-09-15T01:34:37.705533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:46.121349+0000","scrub_duration":0.0021231879999999998,"objects_trimmed":0,"snaptrim_duration":0.00026779800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":147,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T01:34:43.693327+0000","last_change":"2024-09-15T01:34:32.484611+0000","last_active":"2024-09-15T01:34:43.693327+0000","last_peered":"2024-09-15T01:34:43.693327+0000","last_clean":"2024-09-15T01:34:43.693327+0000","last_became_active":"2024-09-15T01:32:23.134394+0000","last_became_peered":"2024-09-15T01:32:23.134394+0000","last_unstale":"2024-09-15T01:34:43.693327+0000","last_undegraded":"2024-09-15T01:34:43.693327+0000","last_fullsized":"2024-09-15T01:34:43.693327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:34:32.484556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:34:32.484556+0000","last_clean_scrub_stamp":"2024-09-15T01:34:32.484556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:37.372594+0000","scrub_duration":0.0017780090000000001,"objects_trimmed":0,"snaptrim_duration":0.00028619499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":150,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T01:34:52.901155+0000","last_change":"2024-09-15T01:34:37.133417+0000","last_active":"2024-09-15T01:34:52.901155+0000","last_peered":"2024-09-15T01:34:52.901155+0000","last_clean":"2024-09-15T01:34:52.901155+0000","last_became_active":"2024-09-15T01:32:23.134685+0000","last_became_peered":"2024-09-15T01:32:23.134685+0000","last_unstale":"2024-09-15T01:34:52.901155+0000","last_undegraded":"2024-09-15T01:34:52.901155+0000","last_fullsized":"2024-09-15T01:34:52.901155+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:34:37.133346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:34:37.133346+0000","last_clean_scrub_stamp":"2024-09-15T01:34:37.133346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:56.178915+0000","scrub_duration":0.0018814260000000001,"objects_trimmed":0,"snaptrim_duration":0.00032938599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":14,"up_from":21,"seq":90194313253,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798140,"kb_used_data":92881488,"kb_used_omap":3959,"kb_used_meta":0,"kb_avail":92881476,"statfs":{"total":95927926784,"available":95110631424,"internally_reserved":12288,"allocated":95110643712,"data_stored":95110643712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4055015,"internal_metadata":0},"hb_peers":[1,2,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":16.196999999999999,"5min":27.753,"15min":29.678999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":299.721,"5min":574.26900000000001,"15min":574.26900000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":18.404,"5min":4.0270000000000001,"15min":1.631},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":342.08300000000003,"5min":342.08300000000003,"15min":342.08300000000003},"last":0.26900000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:33:20 2024","interfaces":[{"interface":"back","average":{"1min":32.810000000000002,"5min":32.810000000000002,"15min":32.810000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":644.97699999999998,"5min":644.97699999999998,"15min":644.97699999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":805.39499999999998}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.71399999999999997,"15min":0.73999999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.69299999999999995,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":11.887,"5min":2.8730000000000002,"15min":1.371},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":214.994,"5min":214.994,"15min":214.994},"last":0.58899999999999997}]},{"osd":5,"last update":"Sun Sep 15 01:34:43 2024","interfaces":[{"interface":"back","average":{"1min":7.8959999999999999,"5min":2.1030000000000002,"15min":1.137},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":159.45500000000001,"5min":159.45500000000001,"15min":159.45500000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.40300000000000002}]},{"osd":6,"last update":"Sun Sep 15 01:34:43 2024","interfaces":[{"interface":"back","average":{"1min":24.809999999999999,"5min":5.4020000000000001,"15min":2.1669999999999998},"min":{"1min":0.38100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":535.32299999999998,"5min":535.32299999999998,"15min":535.32299999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55700000000000005,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75900000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.40999999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82899999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":10.272,"5min":2.625,"15min":1.351},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":173.11099999999999,"5min":173.11099999999999,"15min":173.11099999999999},"last":0.52700000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":6.056,"15min":6.9420000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.589,"5min":134.12200000000001,"15min":134.12200000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":45.521000000000001,"5min":9.7249999999999996,"15min":3.7589999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":846.97400000000005,"5min":846.97400000000005,"15min":846.97400000000005},"last":0.55000000000000004}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.56599999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.5649999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.625},{"interface":"front","average":{"1min":17.137,"5min":4.0940000000000003,"15min":1.9199999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":313.495,"5min":313.495,"15min":313.495},"last":0.61499999999999999}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.60199999999999998}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81399999999999995,"15min":0.81299999999999994},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.503}]}]},{"osd":9,"up_from":21,"seq":90194313250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798144,"kb_used_data":92881484,"kb_used_omap":1555,"kb_used_meta":0,"kb_avail":92881472,"statfs":{"total":95927926784,"available":95110627328,"internally_reserved":12288,"allocated":95110639616,"data_stored":95110639616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592413,"internal_metadata":0},"hb_peers":[0,6,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76700000000000002,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.46999999999999997},"min":{"1min":0.38700000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.37}]},{"osd":6,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":23.486999999999998,"5min":5.2149999999999999,"15min":2.1699999999999999},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":462.93700000000001,"5min":462.93700000000001,"15min":462.93700000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.94399999999999995,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.438}]},{"osd":7,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84399999999999997,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89800000000000002,"5min":1.97,"15min":1.97},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.0720000000000001,"15min":1.1519999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":10.916,"15min":10.916},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.623,"5min":1.218,"15min":1.3169999999999999},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":13.147,"15min":13.147},"last":0.55500000000000005}]},{"osd":10,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":5.8819999999999997,"5min":1.702,"15min":1.006},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":127.033,"5min":127.033,"15min":127.033},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.90600000000000003,"15min":0.95299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92900000000000005,"5min":6.4539999999999997,"15min":6.4539999999999997},"last":0.53400000000000003}]},{"osd":11,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":10.221,"5min":2.5840000000000001,"15min":1.3109999999999999},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":229.28899999999999,"5min":229.28899999999999,"15min":229.28899999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.88700000000000001,"15min":0.93100000000000005},"min":{"1min":0.42199999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.874,"5min":6.9489999999999998,"15min":6.9489999999999998},"last":0.63300000000000001}]},{"osd":12,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":23.875,"15min":27.75},"min":{"1min":0.47899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97699999999999998,"5min":495.95800000000003,"15min":495.95800000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":43.860999999999997,"15min":51.058999999999997},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":914.18499999999995,"15min":914.18499999999995},"last":0.62}]},{"osd":13,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":2.7599999999999998,"5min":1.595,"15min":1.401},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":49.841000000000001,"5min":49.841000000000001,"15min":49.841000000000001},"last":0.59099999999999997}]},{"osd":14,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86099999999999999,"5min":1.96,"15min":1.96},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.68300000000000005}]},{"osd":15,"last update":"Sun Sep 15 01:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98799999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.755},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91000000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69999999999999996}]}]},{"osd":3,"up_from":21,"seq":90194313251,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798148,"kb_used_data":92881480,"kb_used_omap":2136,"kb_used_meta":0,"kb_avail":92881468,"statfs":{"total":95927926784,"available":95110623232,"internally_reserved":12288,"allocated":95110635520,"data_stored":95110635520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2187768,"internal_metadata":0},"hb_peers":[0,2,4,6,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.63},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.365,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.58499999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.91700000000000004}]},{"osd":2,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.57699999999999996,"5min":0.312,"15min":0.312},"max":{"1min":0.99099999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.66300000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.81399999999999995,"15min":0.84699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1850000000000001,"5min":7.4720000000000004,"15min":7.4720000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":7.5629999999999997,"15min":8.7230000000000008},"min":{"1min":0.41099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.1100000000000001,"5min":182.87,"15min":182.87},"last":0.47999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.5,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57299999999999995},{"interface":"front","average":{"1min":28.616,"5min":6.2119999999999997,"15min":2.4780000000000002},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":501.65800000000002,"5min":501.65800000000002,"15min":501.65800000000002},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":24.288,"5min":24.288,"15min":24.288},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":500.678,"5min":500.678,"15min":500.678},"last":0.55100000000000005},{"interface":"front","average":{"1min":35.298999999999999,"5min":35.298999999999999,"15min":35.298999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":500.79899999999998,"5min":500.79899999999998,"15min":500.79899999999998},"last":0.52200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":11.316000000000001,"5min":11.316000000000001,"15min":11.316000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":200.40799999999999,"5min":200.40799999999999,"15min":200.40799999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.496}]},{"osd":11,"last update":"Sun Sep 15 01:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.315},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.35499999999999998}]},{"osd":12,"last update":"Sun Sep 15 01:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.39000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.3300000000000001,"5min":1.762,"15min":1.762},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.177,"5min":1.417,"15min":1.417},"last":0.50900000000000001}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.78400000000000003,"15min":0.76800000000000002},"min":{"1min":0.67900000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.427,"5min":1.637,"15min":1.637},"last":0.44},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0169999999999999,"15min":1.0389999999999999},"min":{"1min":0.63200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.476,"5min":7.4500000000000002,"15min":7.4500000000000002},"last":0.45300000000000001}]},{"osd":14,"last update":"Sun Sep 15 01:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.71099999999999997,"15min":0.68899999999999995},"min":{"1min":0.502,"5min":0.438,"15min":0.438},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.51500000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.59699999999999998}]},{"osd":15,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.37},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.41499999999999998}]}]},{"osd":0,"up_from":21,"seq":90194313252,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798272,"kb_used_data":92881356,"kb_used_omap":5391,"kb_used_meta":0,"kb_avail":92881344,"statfs":{"total":95927926784,"available":95110496256,"internally_reserved":12288,"allocated":95110508544,"data_stored":95110508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5520730,"internal_metadata":0},"hb_peers":[1,3,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.113,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":31.643999999999998,"15min":36.808999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.488,"5min":814.68299999999999,"15min":814.68299999999999},"last":0.751},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.63700000000000001,"15min":0.65200000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1890000000000001,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":40.415999999999997,"5min":8.5380000000000003,"15min":3.2250000000000001},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":799.04999999999995,"5min":799.04999999999995,"15min":799.04999999999995},"last":0.70699999999999996}]},{"osd":6,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69399999999999995,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.2769999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.264,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":46.667999999999999,"5min":9.8330000000000002,"15min":3.694},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":920.58399999999995,"5min":920.58399999999995,"15min":920.58399999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.41599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0629999999999999,"5min":1.304,"15min":1.304},"last":0.76300000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":9,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1499999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.222,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.65100000000000002}]},{"osd":10,"last update":"Sun Sep 15 01:34:40 2024","interfaces":[{"interface":"back","average":{"1min":47.787999999999997,"5min":10.132999999999999,"15min":3.8570000000000002},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":895.93799999999999,"5min":895.93799999999999,"15min":895.93799999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":22.457000000000001,"5min":4.9550000000000001,"15min":2.0379999999999998},"min":{"1min":0.40699999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":414.40899999999999,"5min":414.40899999999999,"15min":414.40899999999999},"last":0.61599999999999999}]},{"osd":12,"last update":"Sun Sep 15 01:34:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.42699999999999999}]},{"osd":13,"last update":"Sun Sep 15 01:34:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":20.651,"15min":23.984000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.98599999999999999,"5min":501.07400000000001,"15min":501.07400000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":35.735999999999997,"15min":41.578000000000003},"min":{"1min":0.52500000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2070000000000001,"5min":501.08800000000002,"15min":501.08800000000002},"last":0.82099999999999995}]},{"osd":15,"last update":"Sun Sep 15 01:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.403,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.85399999999999998}]}]},{"osd":13,"up_from":21,"seq":90194313251,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799356,"kb_used_data":92880272,"kb_used_omap":4493,"kb_used_meta":0,"kb_avail":92880260,"statfs":{"total":95927926784,"available":95109386240,"internally_reserved":12288,"allocated":95109398528,"data_stored":95109398528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4601239,"internal_metadata":0},"hb_peers":[2,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":1.0049999999999999,"15min":1.0660000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.109,"5min":8.1129999999999995,"15min":8.1129999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.997,"15min":1.0600000000000001},"min":{"1min":0.38700000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1739999999999999,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":0.45400000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:33:18 2024","interfaces":[{"interface":"back","average":{"1min":19.710000000000001,"5min":19.710000000000001,"15min":19.710000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":342.56099999999998,"5min":342.56099999999998,"15min":342.56099999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":28.227,"5min":28.227,"15min":28.227},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":495.40800000000002,"5min":495.40800000000002,"15min":495.40800000000002},"last":0.60099999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":24.440999999999999,"5min":5.6139999999999999,"15min":2.476},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":572.322,"5min":572.322,"15min":572.322},"last":0.56699999999999995},{"interface":"front","average":{"1min":41.835999999999999,"5min":9.0950000000000006,"15min":3.6379999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":990.04700000000003,"5min":990.04700000000003,"15min":990.04700000000003},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Sep 15 01:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46999999999999997,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.316},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.58899999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.30399999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:34:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":15.279,"15min":17.722999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.137,"5min":293.34800000000001,"15min":293.34800000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":12.593999999999999,"5min":3.028,"15min":1.4339999999999999},"min":{"1min":0.40100000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":300.29500000000002,"5min":300.29500000000002,"15min":300.29500000000002},"last":0.437}]},{"osd":7,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.91400000000000003,"15min":0.95299999999999996},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.488,"5min":6.4779999999999998,"15min":6.4779999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":21.263000000000002,"5min":5.024,"15min":2.3180000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":493.435,"5min":493.435,"15min":493.435},"last":0.57499999999999996}]},{"osd":8,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.85499999999999998,"15min":0.89300000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90000000000000002,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.624},{"interface":"front","average":{"1min":0.65000000000000002,"5min":2.6000000000000001,"15min":2.9249999999999998},"min":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94999999999999996,"5min":44.985999999999997,"15min":44.985999999999997},"last":0.67800000000000005}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.59599999999999997}]},{"osd":12,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.50800000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.72399999999999998}]},{"osd":14,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73099999999999998,"15min":0.72399999999999998},"min":{"1min":0.56000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.80400000000000005,"15min":0.80800000000000005},"min":{"1min":0.56699999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2669999999999999,"5min":1.474,"15min":1.474},"last":0.55000000000000004}]},{"osd":15,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":46.790999999999997,"5min":9.9949999999999992,"15min":3.8620000000000001},"min":{"1min":0.53400000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":806.28599999999994,"5min":806.28599999999994,"15min":806.28599999999994},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.52500000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2989999999999999,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.749}]}]},{"osd":2,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798848,"kb_used_data":92880780,"kb_used_omap":4400,"kb_used_meta":0,"kb_avail":92880768,"statfs":{"total":95927926784,"available":95109906432,"internally_reserved":12288,"allocated":95109918720,"data_stored":95109918720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4505909,"internal_metadata":0},"hb_peers":[1,3,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95199999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.443}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86599999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.38400000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":46.487000000000002,"5min":9.6280000000000001,"15min":3.4849999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":829.03399999999999,"5min":829.03399999999999,"15min":829.03399999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.43099999999999999,"15min":0.41099999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.36499999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":39.168999999999997,"5min":13.167999999999999,"15min":8.8339999999999996},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":655.90899999999999,"5min":655.90899999999999,"15min":655.90899999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68700000000000006}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0329999999999999,"5min":1.329,"15min":1.329},"last":0.505},{"interface":"front","average":{"1min":54.841000000000001,"5min":11.481,"15min":4.2539999999999996},"min":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":976.00800000000004,"5min":976.00800000000004,"15min":976.00800000000004},"last":0.55300000000000005}]},{"osd":9,"last update":"Sun Sep 15 01:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.46200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82099999999999995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":17.667000000000002,"5min":3.9620000000000002,"15min":1.6779999999999999},"min":{"1min":0.40200000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":340.62599999999998,"5min":340.62599999999998,"15min":340.62599999999998},"last":0.49299999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.68700000000000006,"15min":0.63600000000000001},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":6.4240000000000004,"5min":6.4240000000000004,"15min":6.4240000000000004},"last":0.624},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.74399999999999999,"15min":0.68999999999999995},"min":{"1min":0.47299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":6.4390000000000001,"5min":6.4390000000000001,"15min":6.4390000000000001},"last":0.73499999999999999}]},{"osd":12,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":47.031999999999996,"5min":9.9469999999999992,"15min":3.766},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":834.505,"5min":834.505,"15min":834.505},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97199999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.59899999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99199999999999999,"5min":1.363,"15min":1.363},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.70699999999999996}]},{"osd":14,"last update":"Sun Sep 15 01:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67600000000000005,"15min":0.66200000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0329999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":22.992000000000001,"5min":5.149,"15min":2.1749999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":444.06200000000001,"5min":444.06200000000001,"15min":444.06200000000001},"last":0.69699999999999995}]}]},{"osd":15,"up_from":21,"seq":90194313252,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798352,"kb_used_data":92881288,"kb_used_omap":3203,"kb_used_meta":0,"kb_avail":92881264,"statfs":{"total":95927926784,"available":95110414336,"internally_reserved":24576,"allocated":95110438912,"data_stored":95110438912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3279968,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.79300000000000004,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":12.098000000000001,"5min":2.8300000000000001,"15min":1.2849999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":162.16800000000001,"5min":162.16800000000001,"15min":162.16800000000001},"last":0.70999999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.495},{"interface":"front","average":{"1min":47.701000000000001,"5min":9.8719999999999999,"15min":3.5670000000000002},"min":{"1min":0.26600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":662.84000000000003,"5min":662.84000000000003,"15min":662.84000000000003},"last":388.93400000000003}]},{"osd":4,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64200000000000002,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.63},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57099999999999995,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.59699999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94899999999999995,"5min":1.385,"15min":1.385},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.82099999999999995}]},{"osd":6,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":4.798,"15min":5.5069999999999997},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69999999999999996,"5min":90.775999999999996,"15min":90.775999999999996},"last":0.748},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.69499999999999995}]},{"osd":7,"last update":"Sun Sep 15 01:34:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":40.222999999999999,"15min":46.847000000000001},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73299999999999998,"5min":894.27700000000004,"15min":894.27700000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":39.558,"15min":46.058},"min":{"1min":0.33200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70999999999999996,"5min":878.12199999999996,"15min":878.12199999999996},"last":0.55300000000000005}]},{"osd":8,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":50.171999999999997,"5min":10.541,"15min":3.9359999999999999},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":645.34900000000005,"5min":645.34900000000005,"15min":645.34900000000005},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":27.992000000000001,"15min":32.540999999999997},"min":{"1min":0.45900000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99099999999999999,"5min":542.06600000000003,"15min":542.06600000000003},"last":0.84399999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":1.603,"15min":1.603},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96199999999999997,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.97099999999999997}]},{"osd":14,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83299999999999996,"5min":1.163,"15min":1.163},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.75,"5min":12.249000000000001,"15min":14.164999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.224,"5min":246.56100000000001,"15min":246.56100000000001},"last":0.95099999999999996}]}]},{"osd":4,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798200,"kb_used_data":92881416,"kb_used_omap":3068,"kb_used_meta":0,"kb_avail":92881416,"statfs":{"total":95927926784,"available":95110569984,"internally_reserved":0,"allocated":95110569984,"data_stored":95110569984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3141698,"internal_metadata":0},"hb_peers":[0,3,5,6,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.441,"15min":0.437},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":14.403,"15min":16.731999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60399999999999998,"5min":314.661,"15min":314.661},"last":0.36499999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51300000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.747},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":1.127}]},{"osd":3,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.55900000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0940000000000001,"5min":1.179,"15min":1.179},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.45000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.83299999999999996}]},{"osd":5,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":42.378,"5min":24.170000000000002,"15min":21.135000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":918.75699999999995,"5min":918.75699999999995,"15min":918.75699999999995},"last":0.79100000000000004}]},{"osd":6,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":7.5110000000000001,"5min":2.1309999999999998,"15min":1.2350000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":142.26599999999999,"5min":142.26599999999999,"15min":142.26599999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.57899999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1719999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Sun Sep 15 01:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0840000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.871}]},{"osd":9,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78300000000000003,"15min":0.77900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.93799999999999994}]},{"osd":10,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":40.198999999999998,"5min":8.7089999999999996,"15min":3.4609999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":828.54499999999996,"5min":828.54499999999996,"15min":828.54499999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":14.634,"5min":3.5920000000000001,"15min":1.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":291.08800000000002,"5min":291.08800000000002,"15min":291.08800000000002},"last":0.94799999999999995}]},{"osd":11,"last update":"Sun Sep 15 01:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":28.721,"5min":28.721,"15min":28.721},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":558.56700000000001,"5min":558.56700000000001,"15min":558.56700000000001},"last":0.68200000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1799999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.71299999999999997}]},{"osd":13,"last update":"Sun Sep 15 01:34:33 2024","interfaces":[{"interface":"back","average":{"1min":21.696999999999999,"5min":4.9790000000000001,"15min":2.1930000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":441.02800000000002,"5min":441.02800000000002,"15min":441.02800000000002},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81399999999999995,"15min":0.82099999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.81299999999999994}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":15,"last update":"Sun Sep 15 01:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.61099999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3169999999999999,"5min":1.325,"15min":1.325},"last":0.879},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.90500000000000003,"15min":0.90700000000000003},"min":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.256,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.92300000000000004}]}]},{"osd":5,"up_from":21,"seq":90194313250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798268,"kb_used_data":92881360,"kb_used_omap":3063,"kb_used_meta":0,"kb_avail":92881348,"statfs":{"total":95927926784,"available":95110500352,"internally_reserved":12288,"allocated":95110512640,"data_stored":95110512640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3137455,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78000000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:34:36 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38,"15min":0.379},"min":{"1min":0.25700000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54700000000000004,"5min":0.622,"15min":0.622},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.34000000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:34:36 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.621,"15min":0.60699999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.55600000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.76000000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.628,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.48,"5min":1.524,"15min":1.524},"last":0.629}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.70199999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.504},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.46200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.414999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92200000000000004,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.60799999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":36.902999999999999,"5min":7.8339999999999996,"15min":2.9889999999999999},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":764.33399999999995,"5min":764.33399999999995,"15min":764.33399999999995},"last":0.432},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.44900000000000001}]},{"osd":14,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1899999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":17.335999999999999,"5min":22.786000000000001,"15min":23.693999999999999},"min":{"1min":0.496,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":346.49200000000002,"5min":492.697,"15min":492.697},"last":0.58399999999999996}]},{"osd":15,"last update":"Sun Sep 15 01:34:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":11.316000000000001,"15min":13.064},"min":{"1min":0.54700000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6399999999999999,"5min":291.12599999999998,"15min":291.12599999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.77700000000000002,"15min":0.76100000000000001},"min":{"1min":0.61199999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.68999999999999995}]}]},{"osd":6,"up_from":21,"seq":90194313250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798272,"kb_used_data":92881356,"kb_used_omap":3143,"kb_used_meta":0,"kb_avail":92881344,"statfs":{"total":95927926784,"available":95110496256,"internally_reserved":12288,"allocated":95110508544,"data_stored":95110508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3219025,"internal_metadata":0},"hb_peers":[1,3,4,5,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.85699999999999998,"15min":0.88100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.165,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77600000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.63800000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.878,"5min":1.264,"15min":1.264},"last":0.53400000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.153,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77800000000000002,"5min":1.163,"15min":1.163},"last":0.55500000000000005}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.379,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83799999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72799999999999998,"5min":0.746,"15min":0.746},"last":0.48799999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.78100000000000003}]},{"osd":10,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":61.164000000000001,"5min":12.723000000000001,"15min":4.649},"min":{"1min":0.48099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":656.21000000000004,"5min":656.21000000000004,"15min":656.21000000000004},"last":0.44500000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":11.742000000000001,"5min":2.8580000000000001,"15min":1.3779999999999999},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":200.52799999999999,"5min":200.52799999999999,"15min":200.52799999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.626}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82999999999999996,"5min":1.353,"15min":1.353},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.99199999999999999,"15min":1.036},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97999999999999998,"5min":5.8470000000000004,"15min":5.8470000000000004},"last":0.79700000000000004}]},{"osd":13,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.55800000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.55200000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.034,"5min":1.212,"15min":1.212},"last":0.82899999999999996}]}]},{"osd":7,"up_from":21,"seq":90194313251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798144,"kb_used_data":92881484,"kb_used_omap":2582,"kb_used_meta":0,"kb_avail":92881472,"statfs":{"total":95927926784,"available":95110627328,"internally_reserved":12288,"allocated":95110639616,"data_stored":95110639616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2644083,"internal_metadata":0},"hb_peers":[0,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.48199999999999998,"15min":0.47099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":47.609999999999999,"15min":55.453000000000003},"min":{"1min":0.36699999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.69999999999999996,"5min":874.39499999999998,"15min":874.39499999999998},"last":0.64500000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.58099999999999996,"15min":0.61499999999999999},"min":{"1min":0.26400000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.73399999999999999,"5min":4.7350000000000003,"15min":4.7350000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27800000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.38500000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.42499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.80000000000000004,"15min":0.83799999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89900000000000002,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.77500000000000002}]},{"osd":6,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":1.165,"15min":1.165},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":1.0880000000000001,"15min":1.167},"min":{"1min":0.42799999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":5.6539999999999999,"15min":5.6539999999999999},"last":0.69499999999999995}]},{"osd":8,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82299999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56599999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":10,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":4.4359999999999999,"15min":5.069},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84899999999999998,"5min":62.991999999999997,"15min":62.991999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":12,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67600000000000005,"15min":0.68400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":1.21,"15min":1.21},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":43.418999999999997,"15min":50.549999999999997},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93400000000000005,"5min":792.73599999999999,"15min":792.73599999999999},"last":0.80800000000000005}]},{"osd":13,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.091,"15min":1.157},"min":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92100000000000004,"5min":8.1489999999999991,"15min":8.1489999999999991},"last":0.74299999999999999}]},{"osd":14,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.47199999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.878,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":41.905000000000001,"15min":48.776000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.83099999999999996,"5min":773.18799999999999,"15min":773.18799999999999},"last":0.81899999999999995}]},{"osd":15,"last update":"Sun Sep 15 01:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.1220000000000001,"15min":1.216},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79000000000000004,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83399999999999996,"5min":1.018,"15min":1.018},"last":0.63}]}]},{"osd":8,"up_from":21,"seq":90194313250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798272,"kb_used_data":92881356,"kb_used_omap":3907,"kb_used_meta":0,"kb_avail":92881344,"statfs":{"total":95927926784,"available":95110496256,"internally_reserved":12288,"allocated":95110508544,"data_stored":95110508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4001782,"internal_metadata":0},"hb_peers":[0,1,2,4,6,7,9,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.373},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.626,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.41299999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.497,"15min":0.499},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.625,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":19.939,"15min":23.175000000000001},"min":{"1min":0.38600000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.67300000000000004,"5min":413.66800000000001,"15min":413.66800000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Sun Sep 15 01:34:35 2024","interfaces":[{"interface":"back","average":{"1min":50.540999999999997,"5min":10.541,"15min":3.8740000000000001},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":800.48500000000001,"5min":800.48500000000001,"15min":800.48500000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55100000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.502,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39500000000000002,"15min":0.39600000000000002},"min":{"1min":0.313,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.48899999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.434}]},{"osd":6,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72499999999999998,"5min":1.093,"15min":1.093},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.44700000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.95599999999999996,"5min":1.073,"15min":1.073},"last":0.46400000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":30.347000000000001,"5min":6.5869999999999997,"15min":2.6269999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":505.29300000000001,"5min":505.29300000000001,"15min":505.29300000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":12.156000000000001,"5min":2.9510000000000001,"15min":1.417},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":196.00299999999999,"5min":196.00299999999999,"15min":196.00299999999999},"last":0.66100000000000003}]},{"osd":9,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.46899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97699999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.75600000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51600000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0249999999999999,"5min":1.208,"15min":1.208},"last":0.627},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.004,"5min":1.351,"15min":1.351},"last":0.60199999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.503,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.747}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":1.054,"15min":1.054},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98999999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.73199999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313251,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798144,"kb_used_data":92881484,"kb_used_omap":3285,"kb_used_meta":0,"kb_avail":92881472,"statfs":{"total":95927926784,"available":95110627328,"internally_reserved":12288,"allocated":95110639616,"data_stored":95110639616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3364771,"internal_metadata":0},"hb_peers":[3,5,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.899},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38}]},{"osd":1,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40500000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46100000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75700000000000001,"15min":0.752},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":3.9750000000000001,"5min":3.9750000000000001,"15min":3.9750000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":3.9039999999999999,"5min":4.3150000000000004,"15min":4.383},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":56.188000000000002,"5min":64.938999999999993,"15min":64.938999999999993},"last":0.65700000000000003}]},{"osd":6,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.47699999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":4.7960000000000003,"5min":1.5009999999999999,"15min":0.95199999999999996},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":75.778999999999996,"5min":75.778999999999996,"15min":75.778999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.75900000000000001,"15min":0.78100000000000003},"min":{"1min":0.45300000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84599999999999997,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.503}]},{"osd":8,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.78000000000000003},{"interface":"front","average":{"1min":23.452999999999999,"5min":23.452999999999999,"15min":23.452999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":434.37700000000001,"5min":434.37700000000001,"15min":434.37700000000001},"last":0.77300000000000002}]},{"osd":9,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":44.508000000000003,"15min":51.82},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96299999999999997,"5min":500.92099999999999,"15min":500.92099999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.41399999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.752}]},{"osd":11,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.46200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.69499999999999995}]},{"osd":12,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84199999999999997,"15min":0.87},"min":{"1min":0.503,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0229999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.82999999999999996}]},{"osd":13,"last update":"Sun Sep 15 01:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.4460000000000002},{"interface":"front","average":{"1min":36.435000000000002,"5min":36.435000000000002,"15min":36.435000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":641.649,"5min":641.649,"15min":641.649},"last":2.4740000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.048},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73899999999999999}]},{"osd":15,"last update":"Sun Sep 15 01:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.93899999999999995,"15min":0.97299999999999998},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.036,"5min":6.0670000000000002,"15min":6.0670000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0089999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.79300000000000004}]}]},{"osd":11,"up_from":21,"seq":90194313251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798176,"kb_used_data":92881452,"kb_used_omap":3341,"kb_used_meta":0,"kb_avail":92881440,"statfs":{"total":95927926784,"available":95110594560,"internally_reserved":12288,"allocated":95110606848,"data_stored":95110606848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3421462,"internal_metadata":0},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:33:17 2024","interfaces":[{"interface":"back","average":{"1min":46.469000000000001,"5min":46.469000000000001,"15min":46.469000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":873.95500000000004,"5min":873.95500000000004,"15min":873.95500000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.954,"5min":1.954,"15min":1.954},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":28.431999999999999,"5min":28.431999999999999,"15min":28.431999999999999},"last":0.50900000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.27200000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":13.093,"5min":3.125,"15min":1.4630000000000001},"min":{"1min":0.34100000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":250.767,"5min":250.767,"15min":250.767},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84899999999999998,"5min":0.871,"15min":0.871},"last":0.53700000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.42199999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.84999999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56100000000000005}]},{"osd":8,"last update":"Sun Sep 15 01:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79800000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":6.2300000000000004,"5min":1.671,"15min":0.91100000000000003},"min":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":101.952,"5min":101.952,"15min":101.952},"last":0.68600000000000005}]},{"osd":9,"last update":"Sun Sep 15 01:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":1.1970000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86899999999999999,"5min":1.042,"15min":1.042},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90200000000000002,"5min":2.262,"15min":2.262},"last":0.70199999999999996}]},{"osd":12,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.79000000000000004,"15min":0.81799999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93400000000000005,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.44},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.82099999999999995,"15min":0.84699999999999998},"min":{"1min":0.45300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85999999999999999,"5min":5.3120000000000003,"15min":5.3120000000000003},"last":0.61199999999999999}]},{"osd":13,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":1.903,"5min":0.95599999999999996,"15min":0.79800000000000004},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":25.056999999999999,"5min":25.056999999999999,"15min":25.056999999999999},"last":0.751},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.442,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.78100000000000003}]},{"osd":14,"last update":"Sun Sep 15 01:34:20 2024","interfaces":[{"interface":"back","average":{"1min":4.4009999999999998,"5min":13.781000000000001,"15min":15.343999999999999},"min":{"1min":0.51200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":74.078000000000003,"5min":310.78199999999998,"15min":310.78199999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.436,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97099999999999997,"5min":2.238,"15min":2.238},"last":0.63400000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":12,"up_from":21,"seq":90194313251,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798272,"kb_used_data":92881356,"kb_used_omap":1478,"kb_used_meta":0,"kb_avail":92881344,"statfs":{"total":95927926784,"available":95110496256,"internally_reserved":12288,"allocated":95110508544,"data_stored":95110508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1513934,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":20.391999999999999,"5min":4.4370000000000003,"15min":1.778},"min":{"1min":0.27800000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":480.024,"5min":480.024,"15min":480.024},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.377,"5min":0.435,"15min":0.44500000000000001},"min":{"1min":0.26400000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.68000000000000005,"5min":0.875,"15min":0.875},"last":0.68300000000000005}]},{"osd":1,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":15.561999999999999,"5min":3.8639999999999999,"15min":1.9139999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":360.94400000000002,"5min":360.94400000000002,"15min":360.94400000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.751,"15min":0.79100000000000004},"min":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":3.706,"15min":3.706},"last":0.71999999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.753,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.42999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85099999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.875}]},{"osd":3,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.65800000000000003,"15min":0.67400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80200000000000005,"5min":1.409,"15min":1.409},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60199999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84199999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.0069999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":20.039000000000001,"15min":23.303999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":488.69,"15min":488.69},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.70899999999999996,"15min":0.754},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63300000000000001,"5min":3.5419999999999998,"15min":3.5419999999999998},"last":0.78200000000000003}]},{"osd":6,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.82899999999999996,"15min":0.85499999999999998},"min":{"1min":0.41499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.97599999999999998,"5min":3.6309999999999998,"15min":3.6309999999999998},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.85099999999999998,"5min":1.109,"15min":1.109},"last":0.996}]},{"osd":11,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.64200000000000002,"15min":0.65100000000000002},"min":{"1min":0.46000000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86299999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":32.154000000000003,"5min":7.1440000000000001,"15min":2.976},"min":{"1min":0.46899999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":724.78499999999997,"5min":724.78499999999997,"15min":724.78499999999997},"last":0.93100000000000005}]},{"osd":13,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94399999999999995,"5min":1.145,"15min":1.145},"last":0.97299999999999998},{"interface":"front","average":{"1min":5.3300000000000001,"5min":1.5660000000000001,"15min":0.93799999999999994},"min":{"1min":0.40400000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":107.797,"5min":107.797,"15min":107.797},"last":0.89200000000000002}]},{"osd":14,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.008,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.018,"5min":1.133,"15min":1.133},"last":1.0189999999999999}]},{"osd":15,"last update":"Sun Sep 15 01:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.78300000000000003,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.0409999999999999,"5min":1.712,"15min":1.712},"last":1.034},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.80600000000000005,"15min":0.81799999999999995},"min":{"1min":0.52900000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.995,"5min":1.595,"15min":1.595},"last":1.0600000000000001}]}]},{"osd":1,"up_from":21,"seq":90194313251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798144,"kb_used_data":92881484,"kb_used_omap":3222,"kb_used_meta":0,"kb_avail":92881472,"statfs":{"total":95927926784,"available":95110627328,"internally_reserved":12288,"allocated":95110639616,"data_stored":95110639616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3299624,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":55.167999999999999,"5min":11.429,"15min":4.1390000000000002},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.31},"max":{"1min":875.06100000000004,"5min":875.06100000000004,"15min":875.06100000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.754,"15min":0.754},"last":0.63200000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79100000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.495},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61299999999999999,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91100000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.57199999999999995}]},{"osd":3,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.60199999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.29999999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79200000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.57699999999999996,"5min":28.789000000000001,"15min":33.491},"min":{"1min":0.38400000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82999999999999996,"5min":635.46400000000006,"15min":635.46400000000006},"last":0.79400000000000004}]},{"osd":5,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.53800000000000003},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71199999999999997,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.80500000000000005,"15min":0.85699999999999998},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72699999999999998,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.45200000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":54.337000000000003,"5min":11.387,"15min":4.2290000000000001},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":806.31200000000001,"5min":806.31200000000001,"15min":806.31200000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68000000000000005,"15min":0.68600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83799999999999997,"5min":1.341,"15min":1.341},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.75,"15min":0.751},"min":{"1min":0.52600000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.4379999999999999,"5min":1.825,"15min":1.825},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.48499999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0469999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.63900000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":4.0830000000000002,"5min":1.395,"15min":0.94699999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":55.694000000000003,"5min":55.694000000000003,"15min":55.694000000000003},"last":0.624},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69199999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.88900000000000001,"5min":1.492,"15min":1.492},"last":0.81799999999999995}]},{"osd":10,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":2.149,"15min":2.222},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":17.527000000000001,"5min":27.920000000000002,"15min":27.920000000000002},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":8.1069999999999993,"15min":9.3320000000000007},"min":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.026,"5min":166.16200000000001,"15min":166.16200000000001},"last":0.80800000000000005}]},{"osd":12,"last update":"Sun Sep 15 01:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.496,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84999999999999998,"5min":1.393,"15min":1.393},"last":0.77100000000000002}]}]}],"pool_statfs":[]}} 2024-09-15T01:34:56.529 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T01:34:56.827 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T01:34:56.827 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T01:34:56.827 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T01:34:56.827 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:56.841 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T01:34:56.841 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T01:34:56.841 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:56.895 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T01:34:56.895 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-09-15T01:34:56.895 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:56.948 INFO:tasks.ceph.osd.8:Stopped 2024-09-15T01:34:56.948 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2024-09-15T01:34:56.948 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.001 INFO:tasks.ceph.osd.12:Stopped 2024-09-15T01:34:57.001 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T01:34:57.001 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.016 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T01:34:57.016 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T01:34:57.016 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.071 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T01:34:57.071 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-09-15T01:34:57.071 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.125 INFO:tasks.ceph.osd.9:Stopped 2024-09-15T01:34:57.125 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2024-09-15T01:34:57.126 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.181 INFO:tasks.ceph.osd.13:Stopped 2024-09-15T01:34:57.181 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T01:34:57.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.197 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T01:34:57.197 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-15T01:34:57.197 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.253 INFO:tasks.ceph.osd.6:Stopped 2024-09-15T01:34:57.253 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2024-09-15T01:34:57.254 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.309 INFO:tasks.ceph.osd.10:Stopped 2024-09-15T01:34:57.310 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2024-09-15T01:34:57.310 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.365 INFO:tasks.ceph.osd.14:Stopped 2024-09-15T01:34:57.365 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T01:34:57.365 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.380 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T01:34:57.380 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-15T01:34:57.380 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.436 INFO:tasks.ceph.osd.7:Stopped 2024-09-15T01:34:57.436 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2024-09-15T01:34:57.436 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.491 INFO:tasks.ceph.osd.11:Stopped 2024-09-15T01:34:57.492 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2024-09-15T01:34:57.492 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.546 INFO:tasks.ceph.osd.15:Stopped 2024-09-15T01:34:57.546 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T01:34:57.547 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-15T01:34:57.547 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.570 INFO:tasks.ceph.mgr.y:Stopped 2024-09-15T01:34:57.570 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T01:34:57.570 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.615 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T01:34:57.615 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T01:34:57.615 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T01:34:57.615 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.627 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T01:34:57.627 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T01:34:57.627 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.641 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T01:34:57.641 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T01:34:57.641 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:57.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.11 is failed for ~0s 2024-09-15T01:34:57.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 is failed for ~0s 2024-09-15T01:34:57.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-09-15T01:34:57.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-15T01:34:57.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-15T01:34:57.655 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T01:34:57.655 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T01:34:57.655 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-09-15T01:34:57.704 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T01:34:57.704 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T01:34:57.819 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T01:34:57.819 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T01:34:57.926 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T01:34:57.926 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2024-09-15T01:34:57.985 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T01:34:57.986 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2024-09-15T01:34:58.076 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-15T01:34:58.076 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T01:34:58.188 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-15T01:34:58.188 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T01:34:58.285 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-15T01:34:58.285 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2024-09-15T01:34:58.342 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-15T01:34:58.343 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2024-09-15T01:34:58.402 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi152.front.sepia.ceph.com 2024-09-15T01:34:58.402 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T01:34:58.525 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi152.front.sepia.ceph.com 2024-09-15T01:34:58.525 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-15T01:34:58.579 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@smithi152.front.sepia.ceph.com 2024-09-15T01:34:58.579 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2024-09-15T01:34:58.628 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@smithi152.front.sepia.ceph.com 2024-09-15T01:34:58.628 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2024-09-15T01:34:58.679 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-15T01:34:58.679 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T01:34:58.820 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-15T01:34:58.820 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-15T01:34:58.898 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-15T01:34:58.898 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2024-09-15T01:34:58.957 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-15T01:34:58.957 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2024-09-15T01:34:59.004 INFO:tasks.ceph:Archiving mon data... 2024-09-15T01:34:59.005 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/data/mon.a.tgz 2024-09-15T01:34:59.005 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-09-15T01:34:59.025 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.TnvQoiGVXZ 2024-09-15T01:34:59.025 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.TnvQoiGVXZ 2024-09-15T01:34:59.197 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.TnvQoiGVXZ 2024-09-15T01:34:59.285 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.TnvQoiGVXZ is 394KB 2024-09-15T01:34:59.382 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.TnvQoiGVXZ 2024-09-15T01:34:59.398 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/data/mon.b.tgz 2024-09-15T01:34:59.398 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-15T01:34:59.416 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.TUmhJZH6zF 2024-09-15T01:34:59.416 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.TUmhJZH6zF 2024-09-15T01:34:59.619 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.TUmhJZH6zF 2024-09-15T01:34:59.700 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.TUmhJZH6zF is 554KB 2024-09-15T01:34:59.799 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.TUmhJZH6zF 2024-09-15T01:34:59.814 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/data/mon.c.tgz 2024-09-15T01:34:59.814 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2024-09-15T01:34:59.833 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.WrVtcJFxDm 2024-09-15T01:34:59.833 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.WrVtcJFxDm 2024-09-15T01:35:00.337 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.WrVtcJFxDm 2024-09-15T01:35:00.420 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.WrVtcJFxDm is 554KB 2024-09-15T01:35:00.491 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.WrVtcJFxDm 2024-09-15T01:35:00.505 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T01:35:00.505 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T01:35:00.508 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T01:35:00.510 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T01:35:00.547 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T01:35:00.576 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-15T01:35:00.577 DEBUG:teuthology.orchestra.run.smithi042:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T01:35:00.579 DEBUG:teuthology.orchestra.run.smithi115:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T01:35:00.582 DEBUG:teuthology.orchestra.run.smithi152:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T01:35:00.608 INFO:teuthology.orchestra.run.smithi115.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T01:35:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T01:35:00.611 DEBUG:teuthology.orchestra.run.smithi175:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T01:35:00.636 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T01:35:00.649 INFO:teuthology.orchestra.run.smithi175.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T01:35:00.654 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T01:35:00.656 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/remote/smithi042/crash 2024-09-15T01:35:00.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T01:35:00.686 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/remote/smithi115/crash 2024-09-15T01:35:00.687 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T01:35:00.713 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/remote/smithi152/crash 2024-09-15T01:35:00.713 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T01:35:00.740 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/remote/smithi175/crash 2024-09-15T01:35:00.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T01:35:00.864 INFO:tasks.ceph:Compressing logs... 2024-09-15T01:35:00.865 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T01:35:00.868 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T01:35:00.870 DEBUG:teuthology.orchestra.run.smithi152:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T01:35:00.873 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T01:35:00.893 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T01:35:00.893 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2024-09-15T01:35:00.894 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.13.log 2024-09-15T01:35:00.894 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T01:35:00.894 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.13.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T01:35:00.894 INFO:teuthology.orchestra.run.smithi115.stderr:gzip/var/log/ceph/ceph.audit.log: -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T01:35:00.897 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T01:35:00.897 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mon.b.log: 83.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T01:35:00.897 INFO:teuthology.orchestra.run.smithi115.stderr: 88.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T01:35:00.899 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2024-09-15T01:35:00.899 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T01:35:00.899 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T01:35:00.899 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T01:35:00.899 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.14.log 2024-09-15T01:35:00.900 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-15T01:35:00.900 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-osd.14.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T01:35:00.900 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph.log: 88.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T01:35:00.900 INFO:teuthology.orchestra.run.smithi152.stderr: 83.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T01:35:00.914 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28297.log 2024-09-15T01:35:00.915 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28497.log 2024-09-15T01:35:00.915 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28537.log 2024-09-15T01:35:00.915 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28297.log.gz 2024-09-15T01:35:00.915 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28497.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-15T01:35:00.915 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28497.log.gz 2024-09-15T01:35:00.915 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28457.log 2024-09-15T01:35:00.915 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28537.log.gz 2024-09-15T01:35:00.915 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28337.log 2024-09-15T01:35:00.915 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28417.log 2024-09-15T01:35:00.915 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28457.log.gz 2024-09-15T01:35:00.915 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28337.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.15.log 2024-09-15T01:35:00.915 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28337.log.gz 2024-09-15T01:35:00.916 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28257.log 2024-09-15T01:35:00.916 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28417.log.gz 2024-09-15T01:35:00.916 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.15.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28086.log 2024-09-15T01:35:00.916 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28377.log 2024-09-15T01:35:00.916 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28257.log.gz 2024-09-15T01:35:00.916 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2024-09-15T01:35:00.916 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28086.log.gz 2024-09-15T01:35:00.916 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28377.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28377.log.gz 2024-09-15T01:35:00.916 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28177.log 2024-09-15T01:35:00.916 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28046.log 2024-09-15T01:35:00.916 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28177.log.gz 2024-09-15T01:35:00.916 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T01:35:00.917 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28217.log 2024-09-15T01:35:00.917 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28046.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28046.log.gz 2024-09-15T01:35:00.917 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28736.log 2024-09-15T01:35:00.917 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.29310.log 2024-09-15T01:35:00.917 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28217.log.gz 2024-09-15T01:35:00.917 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28130.log 2024-09-15T01:35:00.917 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28736.log.gz 2024-09-15T01:35:00.917 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.0.29310.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T01:35:00.917 INFO:teuthology.orchestra.run.smithi175.stderr: 56.2% -- replaced with /var/log/ceph/ceph-client.0.29310.log.gz 2024-09-15T01:35:00.917 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28577.log 2024-09-15T01:35:00.917 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28130.log.gz 2024-09-15T01:35:00.918 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28617.log 2024-09-15T01:35:00.918 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28577.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28577.log.gz 2024-09-15T01:35:00.918 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.28617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28617.log.gz 2024-09-15T01:35:00.958 INFO:teuthology.orchestra.run.smithi175.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T01:35:00.977 INFO:teuthology.orchestra.run.smithi152.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T01:35:00.996 INFO:teuthology.orchestra.run.smithi115.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2024-09-15T01:35:01.019 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T01:35:01.019 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42674.log 2024-09-15T01:35:01.019 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42911.log 2024-09-15T01:35:01.019 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41444.log 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42674.log.gz 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41467.log 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42911.log.gz 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42902.log 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41444.log.gz 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29907.log 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41467.log.gz 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44485.log 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42902.log.gz 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44695.log 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29907.log.gz 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34816.log 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44485.log.gz 2024-09-15T01:35:01.020 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42105.log 2024-09-15T01:35:01.021 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44695.log.gz 2024-09-15T01:35:01.021 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29363.log 2024-09-15T01:35:01.021 INFO:teuthology.orchestra.run.smithi042.stderr: 23.5% -- replaced with /var/log/ceph/ceph-client.admin.34816.log.gz 2024-09-15T01:35:01.021 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29282.log 2024-09-15T01:35:01.021 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42105.log.gz 2024-09-15T01:35:01.021 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29363.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.29363.log.gz /var/log/ceph/ceph-client.admin.39822.log 2024-09-15T01:35:01.021 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T01:35:01.021 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32495.log 2024-09-15T01:35:01.021 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29282.log.gz 2024-09-15T01:35:01.021 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40662.log 2024-09-15T01:35:01.021 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39822.log.gz 2024-09-15T01:35:01.021 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40620.log 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr: 22.7% -- replaced with /var/log/ceph/ceph-client.admin.32495.log.gz 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44359.log 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40662.log.gz 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43519.log 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40620.log.gz 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41430.log 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44359.log.gz 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42797.log 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43519.log.gz 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43561.log 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41430.log.gz 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35401.log 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42797.log.gz 2024-09-15T01:35:01.022 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29419.log 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43561.log.gz 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31923.log 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35401.log.gz 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43477.log 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29419.log.gz 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38903.log 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31923.log.gz 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35445.log 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43477.log.gz 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43603.log 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38903.log.gz 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40158.log 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35445.log.gz 2024-09-15T01:35:01.023 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43603.log.gz 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42910.log 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33310.log 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40158.log.gz 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43897.log 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42910.log.gz 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38854.log 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33310.log.gz 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43897.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43897.log.gz -5 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37156.log 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34032.log 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.38854.log.gz 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36790.log 2024-09-15T01:35:01.024 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37156.log.gz 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34032.log.gz 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42097.log 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr: 22.7% -- replaced with /var/log/ceph/ceph-client.admin.36790.log.gz 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32712.log 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33049.log 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42097.log.gz 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33812.log 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32712.log.gz 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29884.log 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33049.log.gz 2024-09-15T01:35:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37222.log 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33812.log.gz 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29884.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29884.log.gz 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37505.log 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30449.log 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr: 22.3% -- replaced with /var/log/ceph/ceph-client.admin.37222.log.gz 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44905.log 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37505.log.gz 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41125.log 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30449.log.gz 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44905.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44905.log.gz 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35957.log 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39990.log 2024-09-15T01:35:01.026 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41125.log.gz 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43225.log 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35957.log.gz 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38969.log 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39990.log.gz 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43225.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.43225.log.gz /var/log/ceph/ceph-client.admin.37775.log 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39388.log 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38969.log.gz 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37775.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.12.log 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37775.log.gz 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34031.log/var/log/ceph/ceph-client.admin.39388.log: 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39388.log.gz 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.12.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40999.log 2024-09-15T01:35:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35842.log 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34031.log.gz 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29860.log 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40999.log.gz 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35842.log: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.35842.log.gz 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29916.log 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41465.log 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29860.log.gz 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29916.log.gz 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41928.log 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35560.log 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41465.log.gz 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41928.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41928.log.gz 2024-09-15T01:35:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30392.log 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44149.log 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35560.log.gz 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30392.log.gz 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42103.log 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30814.log 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44149.log.gz 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42103.log.gz 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42160.log 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39011.log 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30814.log.gz 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42160.log.gz 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36668.log 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38064.log 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.39011.log.gz 2024-09-15T01:35:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37417.log 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr: 28.8% -- replaced with /var/log/ceph/ceph-client.admin.36668.log.gz 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32451.log 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38064.log.gz 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37417.log.gz 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39780.log 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44401.log 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32451.log.gz 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39780.log.gz 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31422.log 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37292.log 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44401.log.gz 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31422.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31422.log.gz 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29915.log 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41460.log 2024-09-15T01:35:01.030 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37292.log.gz 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29915.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29915.log.gz 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37651.log 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38612.log 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41460.log.gz 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37651.log.gz 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39906.log 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41448.log 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38612.log.gz 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39906.log.gz 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40452.log 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43393.log 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41448.log.gz 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40452.log.gz 2024-09-15T01:35:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40746.log 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39738.log 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43393.log.gz 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30371.log 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40746.log.gz 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39738.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39738.log.gz 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30371.log.gz 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34658.log 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33434.log 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34658.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr: 22.3% -- replaced with /var/log/ceph/ceph-client.admin.34658.log.gz 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30456.log 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr: 22.3% -- replaced with /var/log/ceph/ceph-client.admin.33434.log.gz 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29883.log 2024-09-15T01:35:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.30456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42630.log 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30456.log.gz 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29883.log.gz 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29914.log 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39864.log 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42630.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42630.log.gz 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39080.log 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr: 26.3% -- replaced with /var/log/ceph/ceph-client.admin.29914.log.gz 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39864.log.gzgzip 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35915.log 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr: 88.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41083.log 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39080.log.gz 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30564.log 2024-09-15T01:35:01.033 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35915.log.gz 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41083.log.gz 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32023.log 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40830.log 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30564.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30564.log.gz 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38435.log 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32023.log.gz 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32063.log 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40830.log.gz 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33966.log 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38435.log.gz 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39948.log 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr: 23.0% -- replaced with /var/log/ceph/ceph-client.admin.32063.log.gz 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29579.log 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33966.log.gz 2024-09-15T01:35:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29917.log 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39948.log.gz 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31421.log 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29579.log.gz 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29917.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29917.log.gz 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33609.log 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37547.log 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31421.log: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.31421.log.gz 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33609.log.gz 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37589.log 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31824.log 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37547.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37547.log.gz 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30568.log 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37589.log.gz 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37312.log 2024-09-15T01:35:01.035 INFO:teuthology.orchestra.run.smithi042.stderr: 22.3% -- replaced with /var/log/ceph/ceph-client.admin.31824.log.gz 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.30568.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32588.log 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30568.log.gz 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37312.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37312.log.gz 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39591.log 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42631.log 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32588.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32588.log.gz 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43435.log 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39591.log.gz 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36111.log 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42631.log.gz 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39691.log 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43435.log.gz 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36111.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.40957.log 2024-09-15T01:35:01.036 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36111.log.gz 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29919.log 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39691.log.gz 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35124.log 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40957.log.gz 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30552.log 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29919.log.gz 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35330.log 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35124.log.gz 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31049.log 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30552.log.gz 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41450.log 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr: 22.7% -- replaced with /var/log/ceph/ceph-client.admin.35330.log.gz 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34776.log 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31049.log.gz 2024-09-15T01:35:01.037 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40116.log 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41450.log.gz 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42116.log 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34776.log.gz 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31091.log 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40116.log.gz 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42591.log 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42116.log.gz 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31091.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31091.log.gz -5 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40200.log 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41466.log 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42591.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42591.log.gz 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30562.log 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40200.log.gz 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36730.log 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41466.log.gz 2024-09-15T01:35:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33244.log 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30562.log.gz 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36626.log 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36730.log.gz 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42085.log 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33244.log.gz 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31871.log 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36626.log.gz 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42085.log.gz 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34933.log 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32380.log 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31871.log.gz 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43981.log 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34933.log.gz 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38174.log 2024-09-15T01:35:01.039 INFO:teuthology.orchestra.run.smithi042.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.32380.log.gz 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42858.log 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43981.log.gz 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38174.log.gz 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31418.log 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42893.log 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42858.log.gz 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29921.log 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31418.log.gz 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40326.log 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42893.log.gz 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29921.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29921.log.gz 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31175.log 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40242.log 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40326.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40326.log.gz 2024-09-15T01:35:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34556.log 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31175.log.gz 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44527.log 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40242.log.gz 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44065.log 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34556.log.gz 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33133.log 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44527.log.gz 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44065.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.44065.log.gz /var/log/ceph/ceph-client.admin.34718.log 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36552.log 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33133.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33133.log.gz 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31133.log 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34718.log.gz 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33853.log 2024-09-15T01:35:01.041 INFO:teuthology.orchestra.run.smithi042.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.36552.log.gz 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31133.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31133.log.gz 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33853.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27024.log 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.33853.log.gz 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29444.log 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.tmp-client.admin.27024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43309.log 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27024.log.gz 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42043.log 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29444.log.gz 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41820.log 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43309.log.gz 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42786.log 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42043.log: /var/log/ceph/ceph-client.admin.41820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42043.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41998.log 2024-09-15T01:35:01.042 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41820.log.gz 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42786.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42786.log.gz 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40788.log 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41998.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41998.log.gz 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30965.log 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40788.log.gz 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41429.log 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42111.log 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30965.log.gz 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.41429.log: -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.33174.log 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41429.log.gz 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42111.log.gz 2024-09-15T01:35:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34271.log 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44191.log 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr: 22.8% -- replaced with /var/log/ceph/ceph-client.admin.33174.log.gz 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34271.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.42796.log 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34271.log.gz 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44191.log.gz 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41983.log 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33089.log 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42796.log.gz 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37374.log 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41983.log.gz 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43939.log 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33089.log: /var/log/ceph/ceph-client.admin.37374.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33089.log.gz 2024-09-15T01:35:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42511.log 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37374.log.gz 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43939.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43939.log.gz 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38479.log 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42511.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41427.log 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42511.log.gz 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38041.log 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr: 23.9% -- replaced with /var/log/ceph/ceph-client.admin.38479.log.gz 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33374.log 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41427.log.gz 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41456.log 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr: 22.7% -- replaced with /var/log/ceph/ceph-client.admin.38041.log.gz 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30447.log 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33374.log: /var/log/ceph/ceph-client.admin.41456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34136.log 2024-09-15T01:35:01.045 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41456.log.gz 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33374.log.gz 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31279.log 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30447.log.gz 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35999.log 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr: 22.4% -- replaced with /var/log/ceph/ceph-client.admin.34136.log.gz 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31279.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31279.log.gz 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36029.log 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35999.log: 22.7% -- replaced with /var/log/ceph/ceph-client.admin.35999.log.gz 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43813.log 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36029.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36029.log.gz 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32568.log 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43813.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43813.log.gz 2024-09-15T01:35:01.046 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32568.log.gz 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40494.log 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41041.log 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40494.log.gz 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34453.log 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30362.log 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41041.log.gz 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34433.log 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34453.log.gz 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.30362.log: -- /var/log/ceph/ceph-client.admin.29913.log 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30362.log.gz 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34433.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42912.log 2024-09-15T01:35:01.047 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34433.log.gz 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29881.log 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29913.log.gz 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42912.log: -- /var/log/ceph/ceph-client.admin.37837.log 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42912.log.gz 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29881.log.gz 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38364.log 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37837.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42125.log 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr: 92.9% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37837.log.gz 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38364.log: gzip 22.7% -- replaced with /var/log/ceph/ceph-client.admin.38364.log.gz 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43267.log 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42125.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42125.log.gz 2024-09-15T01:35:01.048 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40368.log 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr: 83.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43267.log.gz 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38043.log 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44863.log 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40368.log.gz 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40578.log 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38043.log.gz 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43855.log 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44863.log.gz 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33906.log 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40578.log.gz 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40915.log 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43855.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43855.log.gz 2024-09-15T01:35:01.049 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40284.log 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33906.log.gz 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40410.log 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40915.log.gz 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32873.log 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40284.log: /var/log/ceph/ceph-client.admin.40410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40284.log.gz 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40704.log 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40410.log.gz 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42531.log 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr: 23.1% -- replaced with /var/log/ceph/ceph-client.admin.32873.log.gz 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36318.log 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40704.log.gz 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42531.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42531.log.gz 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41455.log 2024-09-15T01:35:01.050 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36318.log: gzip 22.4% -- replaced with /var/log/ceph/ceph-client.admin.36318.log.gz 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40074.log 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41455.log.gz 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30549.log 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44233.log 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40074.log.gz 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30414.log 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30549.log.gz 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36156.log 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44233.log.gz 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38257.log 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30414.log.gz 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37961.log 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36156.log.gz 2024-09-15T01:35:01.051 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28869.log 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38257.log.gz 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37961.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37961.log.gz 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41452.log 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28869.log.gz 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39649.log 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41452.log.gz 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44107.log 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39649.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39649.log.gz 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36710.log 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44107.log.gz 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37713.log 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36710.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36710.log.gz 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43687.log 2024-09-15T01:35:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37713.log.gz 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31983.log 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30563.log 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43687.log.gz 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44443.log 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31983.log.gz 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44821.log 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30563.log.gz 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40032.log 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44443.log.gz 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44821.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44821.log.gz 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31641.log 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40032.log.gz 2024-09-15T01:35:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39511.log 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31641.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.39512.log 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31641.log.gz 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35487.log 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39511.log.gz 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.39512.log: /var/log/ceph/ceph-client.admin.44737.log 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr: 22.7%/var/log/ceph/ceph-client.admin.35487.log: -- replaced with /var/log/ceph/ceph-client.admin.39512.log.gz 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr: 22.3% -- replaced with /var/log/ceph/ceph-client.admin.35487.log.gz 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44023.log 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44737.log: -- /var/log/ceph/ceph-client.admin.41426.log 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44737.log.gz 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44023.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44023.log.gz 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29537.log 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.41426.log: -- /var/log/ceph/ceph-client.admin.43729.log 2024-09-15T01:35:01.054 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41426.log.gz 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29537.log.gz 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39056.log 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43729.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43729.log.gz 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44569.log 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39056.log.gz 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44611.log 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44569.log.gz 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31007.log 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44611.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42900.log 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44611.log.gz 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34096.log 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31007.log.gz 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44317.log 2024-09-15T01:35:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42900.log: gzip/var/log/ceph/ceph-client.admin.34096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42900.log.gz 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34096.log.gz 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr: -- /var/log/ceph/ceph-client.admin.30572.log 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36511.log 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40873.log 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44317.log.gz 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30572.log.gz 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36511.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29901.log 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36511.log.gz 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32967.log 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40873.log.gz 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28602.log 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29901.log.gz 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31416.log 2024-09-15T01:35:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32967.log: /var/log/ceph/ceph-client.admin.28602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29903.log 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32967.log.gz 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28602.log.gz 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31416.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.43351.log -- replaced with /var/log/ceph/ceph-client.admin.31416.log.gz 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37899.log 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29903.log: /var/log/ceph/ceph-client.admin.43351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40536.log 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29903.log.gz 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43351.log.gz 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37899.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37899.log.gz 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35054.log 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40536.log.gz 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44945.log 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35054.log: 22.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34618.log 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35054.log.gz 2024-09-15T01:35:01.057 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44653.log 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44945.log.gz 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34891.log 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34618.log: gzip/var/log/ceph/ceph-client.admin.44653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34618.log.gz 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44653.log.gz 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr: -- /var/log/ceph/ceph-client.admin.33751.log 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34891.log: /var/log/ceph/ceph-client.admin.41464.log 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34891.log.gz 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33751.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33290.log 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33751.log.gz 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37462.log 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41464.log.gz 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37107.log 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33290.log: /var/log/ceph/ceph-client.admin.37462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39450.log 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37462.log.gz 2024-09-15T01:35:01.058 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33290.log.gz 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37107.log: gzip 22.4% -- replaced with /var/log/ceph/ceph-client.admin.37107.log.gz 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44779.log 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39450.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39450.log.gz 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29879.log 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44779.log.gz 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42153.log 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29879.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29879.log.gz 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44275.log 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42153.log.gz 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31217.log 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44275.log.gz 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30509.log 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31217.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31217.log.gz 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43645.log 2024-09-15T01:35:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30509.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30509.log.gz 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29495.log 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43645.log.gz 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43771.log 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29495.log.gz 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41425.log 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43771.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43771.log.gz 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29910.log 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41425.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42141.log 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41425.log.gz 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42138.log 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29910.log.gz 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42141.log.gz/var/log/ceph/ceph-client.admin.42138.log: 2024-09-15T01:35:01.060 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42138.log.gz 2024-09-15T01:35:01.061 INFO:teuthology.orchestra.run.smithi042.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.12.log.gz 2024-09-15T01:35:01.061 INFO:teuthology.orchestra.run.smithi115.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T01:35:01.064 INFO:teuthology.orchestra.run.smithi152.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2024-09-15T01:35:01.073 INFO:teuthology.orchestra.run.smithi152.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T01:35:01.089 INFO:teuthology.orchestra.run.smithi175.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-15T01:35:01.090 INFO:teuthology.orchestra.run.smithi175.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.15.log.gz 2024-09-15T01:35:01.095 INFO:teuthology.orchestra.run.smithi115.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T01:35:01.097 INFO:teuthology.orchestra.run.smithi115.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T01:35:01.117 INFO:teuthology.orchestra.run.smithi152.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-15T01:35:01.131 INFO:teuthology.orchestra.run.smithi152.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.14.log.gz 2024-09-15T01:35:01.133 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-09-15T01:35:01.133 INFO:teuthology.orchestra.run.smithi152.stderr:real 0m0.249s 2024-09-15T01:35:01.134 INFO:teuthology.orchestra.run.smithi152.stderr:user 0m0.864s 2024-09-15T01:35:01.134 INFO:teuthology.orchestra.run.smithi152.stderr:sys 0m0.048s 2024-09-15T01:35:01.143 INFO:teuthology.orchestra.run.smithi175.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T01:35:01.180 INFO:teuthology.orchestra.run.smithi042.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T01:35:01.184 INFO:teuthology.orchestra.run.smithi042.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T01:35:01.198 INFO:teuthology.orchestra.run.smithi115.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.13.log.gz 2024-09-15T01:35:01.200 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-15T01:35:01.200 INFO:teuthology.orchestra.run.smithi115.stderr:real 0m0.317s 2024-09-15T01:35:01.200 INFO:teuthology.orchestra.run.smithi115.stderr:user 0m0.928s 2024-09-15T01:35:01.200 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m0.045s 2024-09-15T01:35:01.230 INFO:teuthology.orchestra.run.smithi042.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2024-09-15T01:35:01.241 INFO:teuthology.orchestra.run.smithi175.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2024-09-15T01:35:01.243 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-09-15T01:35:01.243 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m0.354s 2024-09-15T01:35:01.243 INFO:teuthology.orchestra.run.smithi175.stderr:user 0m0.980s 2024-09-15T01:35:01.243 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m0.054s 2024-09-15T01:35:01.307 INFO:teuthology.orchestra.run.smithi042.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T01:35:01.309 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T01:35:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:real 0m0.427s 2024-09-15T01:35:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:user 0m1.355s 2024-09-15T01:35:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m0.224s 2024-09-15T01:35:01.310 INFO:tasks.ceph:Archiving logs... 2024-09-15T01:35:01.310 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/remote/smithi042/log 2024-09-15T01:35:01.311 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T01:35:01.778 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/remote/smithi115/log 2024-09-15T01:35:01.779 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T01:35:01.940 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/remote/smithi152/log 2024-09-15T01:35:01.940 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T01:35:02.095 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/remote/smithi175/log 2024-09-15T01:35:02.096 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T01:35:02.277 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T01:35:02.287 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T01:35:02.287 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T01:35:02.290 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T01:35:02.292 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T01:35:02.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T01:35:02.412 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T01:35:02.421 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T01:35:02.421 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:35:02.424 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:35:02.427 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:35:02.429 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:35:02.439 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-09-15T01:35:02.441 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-09-15T01:35:02.442 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-15T01:35:02.444 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T01:35:02.444 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-09-15T01:35:02.445 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 10 -93us[ -93us] +/- 63ms 2024-09-15T01:35:02.445 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv02.front.sepia.ceph.com 3 6 377 10 +161us[ +160us] +/- 63ms 2024-09-15T01:35:02.445 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -140us[ -140us] +/- 85ms 2024-09-15T01:35:02.445 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:35:02.445 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-09-15T01:35:02.446 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T01:35:02.447 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-09-15T01:35:02.447 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 12 -68us[ -61us] +/- 63ms 2024-09-15T01:35:02.447 INFO:teuthology.orchestra.run.smithi152.stdout:^* hv02.front.sepia.ceph.com 3 6 377 9 +169us[ +177us] +/- 63ms 2024-09-15T01:35:02.447 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -121us[ -114us] +/- 85ms 2024-09-15T01:35:02.447 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:35:02.448 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T01:35:02.448 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-15T01:35:02.448 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 9 -75us[ -75us] +/- 63ms 2024-09-15T01:35:02.448 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 +168us[ +168us] +/- 63ms 2024-09-15T01:35:02.448 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -107us[ -107us] +/- 85ms 2024-09-15T01:35:02.448 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:35:02.449 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T01:35:02.449 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-09-15T01:35:02.449 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 12 -69us[ -72us] +/- 63ms 2024-09-15T01:35:02.449 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv02.front.sepia.ceph.com 3 6 377 9 +175us[ +172us] +/- 63ms 2024-09-15T01:35:02.449 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -120us[ -123us] +/- 85ms 2024-09-15T01:35:02.449 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:35:02.450 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T01:35:02.459 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T01:35:02.459 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T01:35:02.466 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T01:35:02.566 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T01:35:02.638 DEBUG:teuthology.orchestra.run.smithi152:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T01:35:02.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.11 has been restored 2024-09-15T01:35:02.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 has been restored 2024-09-15T01:35:02.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-09-15T01:35:02.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-15T01:35:02.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-15T01:35:02.703 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T01:35:02.773 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T01:35:02.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:35:02.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T01:35:02.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:35:02.838 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T01:35:02.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:35:02.866 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T01:35:02.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:35:02.896 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T01:35:02.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T01:35:02.914 INFO:teuthology.task.internal:Duration was 570.437807 seconds 2024-09-15T01:35:02.915 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T01:35:02.923 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T01:35:02.924 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T01:35:02.926 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T01:35:02.929 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T01:35:02.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T01:35:02.968 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T01:35:02.970 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T01:35:02.972 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T01:35:02.977 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T01:35:03.357 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T01:35:03.357 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T01:35:03.357 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T01:35:03.382 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-09-15T01:35:03.382 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T01:35:03.409 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2024-09-15T01:35:03.409 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T01:35:03.431 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-09-15T01:35:03.431 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T01:35:03.460 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T01:35:03.460 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T01:35:03.463 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T01:35:03.465 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T01:35:03.474 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T01:35:03.529 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T01:35:03.529 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T01:35:03.534 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T01:35:03.537 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T01:35:03.540 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T01:35:03.675 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T01:35:03.685 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T01:35:03.685 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T01:35:03.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T01:35:03.737 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T01:35:03.761 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T01:35:03.786 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T01:35:03.796 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:35:03.798 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:35:03.801 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:35:03.804 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:35:03.820 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-09-15T01:35:03.822 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-09-15T01:35:03.825 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2024-09-15T01:35:03.849 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-09-15T01:35:03.869 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:35:03.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:35:03.901 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:35:03.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:35:03.915 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:35:03.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:35:03.930 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:35:03.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:35:03.944 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T01:35:03.955 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T01:35:03.955 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/remote/smithi042 2024-09-15T01:35:03.956 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T01:35:03.991 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/remote/smithi115 2024-09-15T01:35:03.991 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T01:35:04.025 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/remote/smithi152 2024-09-15T01:35:04.025 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T01:35:04.058 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905045/remote/smithi175 2024-09-15T01:35:04.058 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T01:35:04.098 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T01:35:04.098 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T01:35:04.101 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T01:35:04.104 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T01:35:04.107 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T01:35:04.141 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T01:35:04.153 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T01:35:04.153 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T01:35:04.170 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T01:35:04.170 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T01:35:04.172 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T01:35:04.174 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T01:35:04.177 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T01:35:04.185 INFO:teuthology.orchestra.run.smithi042.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 01:35 /home/ubuntu/cephtest 2024-09-15T01:35:04.187 INFO:teuthology.orchestra.run.smithi115.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 01:35 /home/ubuntu/cephtest 2024-09-15T01:35:04.189 INFO:teuthology.orchestra.run.smithi152.stdout: 265776 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 01:35 /home/ubuntu/cephtest 2024-09-15T01:35:04.195 INFO:teuthology.orchestra.run.smithi175.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 01:35 /home/ubuntu/cephtest 2024-09-15T01:35:04.196 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T01:35:04.206 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T01:35:04.230 INFO:teuthology.run:Summary data: description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} duration: 570.4378068447113 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-15T01:35:04.230 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T01:35:04.316 INFO:teuthology.run:pass